On Monday, privately held J. Crew confirmed that 72-year-old Drexler will step down in a decision that he said was his own. “I am relaxed-ish,” Drexler told WWD, which had an exclusive interview about the change at the top of the retail company. He will be succeeded in July by James Brett, currently president of Williams Sonoma’s (WSM) furniture retailer West Elm, WWD reports. Drexler, who will stay on as chairman, invested $100 million into J. Crew and owns 10% of the company.

“Jim has a proven track record of pushing for innovation and growing omni-channel brands,” said Drexler in a statement. “I look forward to moving into my new role and assist Jim and the team in every way possible to help ensure a smooth and successful transition.”

Drexler first joined J. Crew in 2003 after he was abruptly fired from rival Gap (GPS) a year earlier. He also previously served on Apple’s (AAPL) board, but retired in 2015.

The change at the top of J. Crew comes as the apparel retailer—which for decades has been closely associated with classic American preppy fashions—has reported sales declines at the namesake brand that haven’t fully been offset by the growing sales for its newer sibling brand Madewell. Last year, for example, sales at J. Crew dropped 6% to $2.02 billion while Madewell’s sales increased 14% to $341.6 million. J. Crew has generated major headlines of late but for all the wrong reasons: it announced job cuts in April, a dicey debt restructuring in May, saw the high-profile exit of creative director Jenna Lyons, and shuffled other key executive posts this spring.

J. Crew ultimately has found itself in an untenable position in the apparel industry that has hurt the likes of Gap and Ralph Lauren (RL)—a slice of apparel that can’t compete with the cheap, fast fashions churned out by the likes of H&M and Zara. Brands like J. Crew have also faced problematic in-store traffic trends as more shopping gravitates to online channels. While J. Crew has a website and was originally known as a catalog business, it still depends on hundreds of retail stores to book sales. That’s problematic when customers are visiting those stores less frequently. E-commerce built brands are more nimble as it relates to marketing and distributing to newer, web-savvy millennial customers.

Meanwhile, fast-fashion brands have been able to churn out new styles at a quick pace that defies how apparel was traditionally rolled out each season. That puts brands like J. Crew in a difficult position to compete as styles come and go quicker than ever. J. Crew has also received criticism for pricing its stylish items too high, especially in a market where fast fashion comes cheap.

Drexler, who won a lot of praise for making Gap a leader in America’s malls, had recently acknowledged that J. Crew had made some mistakes. In a recent interview with the Wall Street Journal, he conceded he didn’t catch on fast enough to just how quickly fashions can change thanks to social media. He said J. Crew had become too elitist and would need to transition to both lower prices and a more accessible image.

Under Drexler’s leadership, J. Crew soared in sales and even went public in 2006 in a splashy debut that raised $376 million on the stock market. J. Crew then agreed to be taken private in a $3 billion deal with two investment firms in 2010—a transaction that left it with a large pile of debt that’s coming due at a time when sales are slowing and many brick-and-mortar retailers find themselves landing in bankruptcy as spending gravitates to online channels.