Labour MP Dawn Butler led calls for the billionaire businessman to be disciplined by TV bosses after he posted the picture of the Senegalese World Cup squad which had been edited to include a picture of handbags and sunglasses laid out on sheets.
In the now-deleted tweet, Lord Sugar wrote: "I recognise some of these guys from the beach in Marbella. Multi tasking resourceful chaps."
In a subsequent tweet, he said he thought his post was 'funny'.
He posted: "I cant see what I have to apologise for.... you are OTT... its a bloody joke", before later issuing a grovelling apology.
Ms Butler, Labour MP for Brent Central, has said the BBC, which airs Lord Sugar's reality programme The Apprentice, should take "swift action" against him.
She wrote, referring to The Apprentice star's common phrase: "Take a look at the BBC's Code of Conduct. It's time the @BBC pulled @Lord-Sugar into the boardroom and told him 'You're Fired'."
Ms Butler had earlier tweeted that she was "very troubled after seeing @Lord-Sugar racist tweet".
She added: "I will be writing to the House of Lords Commissioner for Standards and the @BBC calling for an immediate investigation. Racism has no place in Parliament or society.
"Swift action must be taken."
I misjudged me earlier tweet. It was in no way intended to cause offence, and clearly my attempt at humour has backfired. I have deleted the tweet and am very sorry.— Lord Sugar (@Lord_Sugar) June 20, 2018
Actress Kelechi Okafor tweeted: "Let me tell you why @Lord_Sugar is a wasteman. He’s one of those wilfully ignorant people who thinks he can’t be racist because he grew up poor.
"The tweet is overtly racist but he has said with his chest that he has nothing to apologise for."
Lord Sugar has acknowledged this was a seriously misjudged tweet, and he’s in no doubt about our view on this. It’s right he’s apologised unreservedly.— BBC Press Office (@bbcpress) June 20, 2018
BBC World News anchor Babita Sharma said it was a "shocking, vile tweet that you take a screen grab of because you know it will soon be deleted."
Lord Sugar eventually apologised for the tweet and deleted messages in which he had defended himself.
A shocking, vile tweet that you take a screen grab of because you know it will soon be deleted. pic.twitter.com/28E6SxEva7— Babita Sharma (@BabitaBBC) June 20, 2018
He wrote: "I misjudged me earlier tweet. It was in no way intended to cause offence, and clearly my attempt at humour has backfired. I have deleted the tweet and am very sorry."
The original tweet showed all 11 members of the team that beat Poland 2-1 in Russia last night, including Liverpool star Sadio Mané and Everton’s Idrissa Gueye.
Senegal are ranked 27th in the world in official FIFA rankings.
Critics urged people to complain to the House of Lords Commissioner for Standards about the tweet.
Lord Sugar sits as a crossbench peer in the House of Lords after quitting the Labour Party in 2015, saying he was "disillusioned" at the direction of the party.
The latest season of the BBC's hit show The Apprentice is due to air later this year. The US version of the show has previously been hosted by now-US President Donald Trump.
A BBC spokesman said: "Lord Sugar has acknowledged this was a seriously misjudged tweet, and he’s in no doubt about our view on this. It’s right he’s apologised unreservedly."