Kelly, 52, is charged with sex trafficking in New York and attempting to influence a case in Atlanta, and was arrested while walking his dog, a source with knowledge of the arrest said.
A five-count indictment against Kelly will be unsealed Friday in the Eastern District of New York, a source with that district's US attorney's office told CNN. Kelly will appear in federal court in Brooklyn at a later date, the source said.
His arrest comes after a 13-count indictment was returned Thursday in the Northern District of Illinois that includes child pornography and obstruction of justice charges, said Joseph Fitzpatrick, assistant US attorney for the Northern District of Illinois.
The New York Police Department, Homeland Security Investigations and other federal partners assisted in Thursday's arrest, a law enforcement source said.
He's faced abuse allegations for decades
Until Thursday, Kelly was free on a $1 million bond set in February.
He has faced accusations of abuse, manipulations and inappropriate encounters with girls and young women for more than two decades.
He was charged in February with 10 counts of aggravated criminal sexual abuse involving four alleged victims, including three who prosecutors say were underage girls. The charges span from 1998 to 2010. Kelly pleaded not guilty.
He posted the $100,000 bail and left Chicago's Cook County Jail less than three days after he was arrested. He was arrested and jailed again in March for failure to pay his ex-wife child support of $161,000.
In late May, a grand jury indicted him on 11 more charges pertaining to one of the four accusers, including aggravated criminal sexual assault, a Class X felony; and criminal sexual assault, a Class 1 felony. A person convicted of Class X felonies can get up to 30 years in prison under Illinois law.
Kelly has vehemently denied any allegations of sexual misconduct.