The board dumped Charney last week citing misuse of company money and his part in spreading a nude photo of an ex-employee who had sued him.

Among the accusations, the board said Charney was aware of “but took no steps to prevent” an employee under his direct supervision from creating “false, defamatory and impersonating blog posts about former American Apparel employees.” He also “presented significant severance packages” to multiple former employees to ensure that his “misconduct vis-a-vis these employees would not subject [him] to personal liability,” according to his dismissal letter that was made public.

Charney’s lawyer sent a letter to the board on Thursday threatening legal action if the board did not reinstate him in his executive roles.

Lawyer Patricia Glaser said American Apparel’s (APP) board did not provide the mandatory 21 days’ notice of his severance package as is required by law. Charney’s firing therefore is “not only unconscionable but illegal,” the letter said.

The former CEO maintains a 27% ownership of American Apparel shares and said that he “believes that such termination is without merit and intends to contest it vigorously,” according to a U.S. Securities and Exchange Commission filing submitted Monday.

The board does not see a meeting being worthwhile at this point, a person with knowledge of the situation told Reuters, although the board would consider an agreement with Charney, whose last day at the company is July 18, if he were to promise not to challenge the board’s leadership.

American Apparel’s board is expected to announce its hiring of Peter J. Solomon soon, the source said. The exact role the bank would play is still unclear, but it may provide financing alternatives.

The $120 million fashion brand has struggled with falling sales and heavy debt over the past few years. The retailer’s debt burden is now more than $250 million.