Owners of the New Romantic Ink Tattoo Studio in Wolverhampton removed the sign within hours, but not before it was photographed and put on Twitter and prompted a visit from council officers.
It was put up after an Iranian man allegedly made threats to kill and was arrested by police.
The owners stressed they were not banning foreign customers but had become concerned that they could put permanent markings on people's bodies they did not want if there was a breakdown in communication.
Tattooist Dan Flavell said there had been 'death threats' from an Iranian man who had had a tattoo and had disputed the price before bringing his friends into the parlour with him.
The sign read: "If you can't speak English don't even bother coming in."
Mr Flavell, who is in his mid 20s and runs the Berry Street shop with girlfriend April Gibson, said he took the sign down a few hours later after worrying it could be 'misconstrued'.
He said: "We're not racist. We would never dream of saying we won't tattoo people because of their race or where they may be from.
"The reason for the sign is tattoos are permanent and we cannot take the risk of making a mistake because we cannot communicate with a customer.
"We had death threats on Wednesday when an Iranian man came in. The price was £90 for an tattoo he wanted on his back. But he only wanted to pay £70. He became very aggressive then got three friends who were waiting in a car outside to come in.
"He was threatening to kill us. We called the police and he was arrested.
"That's when I wrote the sign. But then I took it down because I was worried something like this would happen and that some people might not understand what I meant by it.
But a university lecturer who spotted the sign said on Twitter he thought it was racist.
Dr Aidan Byrne, a senior lecturer in English and media and cultural studies at the University of Wolverhampton who tweets as @PlashingVole, said: "If someone cannot understand English they would not have been able to read the sign."
And Councillor Roger Lawrence, leader of Wolverhampton City Council, said: "I'm glad the sign was taken down. It's a throwback to the 1960s."
West Midlands Police confirmed a 25-year-old man was arrested on suspicion of making threats to kill 'following a dispute in a business premises' in Berry Street.
Paul Uppal, MP for Wolverhampton South West said: "The sign was a backwards step but at least it's down now."
Andy Jervis, Wolverhampton City Council's head of regulatory services, said: "While we are pleased that the sign has been voluntarily removed, our officers have visited the business concerned and stressed that this sort of behaviour is totally unacceptable.
"Not only is this damaging to the reputation of the business, but it is also damaging to the reputation of the city which is very much a tolerant and multi-cultural one."