Peter Griffiths, the former Conservative MP for Smethwick, has died at the age of 85.
He was the MP for just two years from 1964 to 1966 but was accused by the then Labour prime minister Harold Wilson of having run an 'utterly squalid' campaign.
Mr Griffiths always denied saying 'if you want a n***** for a neighbour, vote Labour' but it followed him around for the rest of his career.
The Tories lost Smethwick in 1966 but Mr Griffiths went on to become MP for Portsmouth North in 1979 until the landslide Labour victory of 1997.
His controversial election in Smethwick came as the issue of immigration in the 1960s had driven a wedge between workers in the town and the then Labour MP, Patrick Gordon Walker.
Home-made placards using the slogan were made by people supporting Mr Griffiths, although he said he had nothing to do with them.
He was criticised for not condemning the placards and for saying: "I should think that is a manifestation of the popular feeling. I would not condemn anyone who said that. I would say that is how people see the situation in Smethwick. I fully understand the feelings of the people that say it. I would say it is exasperation, not fascism."
While he distanced himself from the far right, Mr Griffiths also supported the former Smethwick Council when it sought to buy up a row of houses to let them exclusively to white families.
After losing the Smethwick seat he wrote a book, A Question of Colour?.
In it he blamed the spread of disease on immigrants and praised apartheid South Africa as a model of democracy. Mr Wilson had criticised the then Tory leader Sir Alec Douglas-Home for not disowning Mr Griffiths.
He said: "If Sir Alec does not take what I am sure is the right course, Smethwick Conservatives can have the satisfaction of having sent a member who, until another election returns him to oblivion, will serve his time as a parliamentary leper."
Twenty Tory MPs then walked out of the chamber and Mr Griffiths replied: "If Harold Wilson wants a fight, then he has come to the right man."
Peter Griffiths was born in West Bromwich in 1928, studied at City of Leeds College and the University of Birmingham and became a Smethwick councillor in 1955. He was also head teacher of Hall Green Primary School in West Bromwich.
In between his defeat in Smethwick and victory in Portsmouth he lectured at Portsmouth College of Technology.
Paul Uppal, Conservative MP for Wolverhampton South West, said: Peter Griffiths was a Conservative MP for 20 years and my thoughts go out to his friends and family at this difficult time.
I am glad to say that both Britain and the Conservative Party have come an extremely long way since that infamous campaign in Smethwick in 1964.
At the last General Election, the number of Conservative MPs from an ethnic minority more than quadrupled going from 2 to 11 of which I was one."