The arrests were made as part of an FBI investigation into alleged healthcare fraud and union corruption that relied on recorded conversations.
NBC4 reports that Andrew "Mush" Russo, the 87-year-old alleged boss of the Colombo crime family, and an underboss, Benjamin "Benji" Castellazzo, were among those arrested.
The FBI is still seeking one more defendant, who is still at large.
Of the dozen people arrested, nine are suspected members of the Colombo crime family, and one is allegedly a soldier in the Bonanno crime family.
One of the alleged Colombo mobsters, Vincent Ricciardo, allegedly threatened to kill a man who would not comply with his demands.
"I'll put him in the ground right in front of his wife and kids, right in front of his f****** house," he said in a recorded conversation. "I'm not afraid to go to jail ... I would f****** shoot him right in front of his wife and kids. Call the police, f*** it, let me go, how long you think I'm gonna last anyway?"
According to federal agents the Colombo crime family is "thriving and continues to engage in various crimes including, among others, extortion, loansharking, money laundering, fraud and drug trafficking."
Mr Russo is no stranger to prison. Before his most recent arrest in 2011, recorded conversations show the alleged boss has no intention of leaving the mob.
"I can't walk away, I can't rest." he said.
He also once admitted in a recorded conversation that he has "never hesitated" to use violence to discipline mobsters.
Mr Russo has previous bribery, racketeering and witness tampering convictions. He was also charged for ordering the killing of a Colombo underboss, Joseph Scopo, during a civil war within the family in 1993.
The defendants in the case were arraigned on Tuesday afternoon in Brooklyn Federal Court, though the exact charges were not immediately made public. They are expected to be charged for alleged labor racketeering.
The Colombo crime family is the youngest of New York's infamous "five families”, which include the Colombo, Bonanno, Gambino, Genovese and Lucchese crime families.
The Five Families were formed after Charles "Lucky" Luciano created the "Mafia Commission" following a bloody war in 1930 between competing mob clans.