The Conservative MP for Lichfield posted a tweet accusing Miqdaad Versi, a spokesperson for the Muslim Council of Britain (MCB), of damaging "Anglo-Muslim relations".
It came after Mr Versi claimed there were "literally hundreds" of examples of Islamophobia in the Conservative Party.
In response Mr Fabricant tweeted: "Your spite and unpleasantness neither does the cause of tolerance in this country nor the cause of Anglo-Muslim relations any good at all.”
He later deleted the tweet and posted another one referring to "relations between Muslims and non-Muslims".
Mr Fabricant had pointed out that unlike Labour, the Conservatives have never been investigated by the Equality and Human Rights Commission (EHRC).
A Conservative Party spokesperson, said: “Michael Fabricant has clarified his tweet. He is raising an important point about how it is the Labour Party that has been investigated by the EHRC for racism.
"The Conservative Party does not tolerate prejudice and discrimination.”
Mr Fabricant said: “I am clear that there is no connection between religion and nationality, which is why I have clarified my tweet accordingly.”
He said he had deleted his original tweet because "some people chose to misinterpret it".
A spokesman for the MCB claimed Mr Fabricant had implied that "Muslims are foreign and alien to the English" and called for the MP to face an investigation over his tweet.
He added that the comment “be it inadvertent or not, betrays a view of Muslims institutionalised in sections of the Conservative Party - that Muslims are not part and parcel of British society”.
The MCB has previously complained to the EHRC over claims of Islamaphobia in the Conservative Party. No action has ever been taken.
Labour was last week found to have breached equality laws by the same body over its failure to deal with anti-Semitism under former leader Jeremy Corbyn.
It comes after a new survey found that more than half of Labour's Muslim members do not believe that new leader Sir Keir Starmer is able to deal with Islamophobia in the party.
Research by the Labour Muslim Network found that 55 per cent of respondents said they do not "trust the leadership of the Labour Party to tackle Islamophobia effectively", while a quarter of respondents said they had directly experienced Islamophobia within the party.