The withdrawal comes after a week of scrutiny of decades-old abuse allegations from Puzder’s ex-wife, which she has since retracted.

Lisa Henning, who now goes by Lisa Fierstein, alleged in 1988 divorce filings that Puzder physically assaulted her in 1986, resulting in “severe and permanent” injuries including “bruises and contusions to the chest, back, shoulders and neck” and two ruptured discs that caused her to seek medical attention, according to records of the divorce obtained by TIME.

Henning also appeared in disguise to discuss her experience in a 1990 Oprah episode about domestic violence. Henning said her ex-husband (whom she does not name in the tape) “vowed revenge” when she made the abuse allegations public, according to a copy of the tape obtained by POLITICO.

“Remember my ex-husband was a public figure. Everyone knew him and knew what he was doing,” Henning said in the Oprah interview. “And once I made that public, he vowed revenge, he said ‘I’ll see you in the gutter, this will never be over, you will pay for this.'”

Puzder also came under scrutiny last week for employing an undocumented immigrant to clean his home. He said in a statement that he had not known the housekeeper was undocumented, and that once he found out, he and his wife immediately fired her and offered her help to get legal status.

Puzder has been widely criticized by unions and worker’s rights organizations. Workers at Carl’s Jr. and Hardees, which are operated by Puzder-run CKE restaurants, filed 33 state and federal complaints against their employer for wage theft, sexual harassment, unfair overtime practices, among other things.

The nomination process was also derailed by Puzder’s attempts to disentangle his financial assets from anything that might constitute a conflict of interest. In order to serve, Puzder would have to step down as CEO of CKE Restaurants and divest from more than 200 companies and funds. Puzder delayed filing the necessary financial paperwork for weeks.

He has also voiced support for replacing workers with robots: “They’re always polite, they always upsell, they never take a vacation, they never show up late, there’s never a slip-and-fall, or an age, sex, or race discrimination case,” he told Business Insider in 2016.

Puzder also had a long record of supporting immigration reform, which puts him at-odds with an administration that’s advocating for a tougher approach.

This article originally appeared on Time.com.