The fashion brand, which this week announced a $400 million comeback plan that will reportedly include closing 50 stores and cutting 1,000 jobs, gets 24% of its annual sales from department stores, with Macy’s (M) by far its biggest account.

That means $1.8 billion of Ralph Lauren’s annual sales come from stores like Macy’s, Nordstrom (JWN), and Belk, which are all struggling with quarter after quarter of declining same-store sales, creating a domino effect on the fashion brand.

Macy’s last month reported comparable sales (including shops it licenses to outside brands) fell a worse-than-expected 5.6% in the first fiscal quarter, for their fifth straight decline as the department store blamed “the uncertain direction of consumer spending.” And Nordstrom did only slightly better.

So it’s easy to see why some many Ralph Lauren items end up at clear-out stores like TJX’s (TAX) T.J. Maxx, where they sell for far less, which over time ruins a brand’s cachet on top of yielding lower top-line sales.

CEO Stefan Larsson said at Ralph Lauren’s first-ever analyst day this week that heavy discounting is leading the company to reduce shipments to wholesalers like department stores, with revenue in that wholesale channel set to fall by a double-digit percentage in 2017.

Larsson, who steered the comeback of Gap Inc’s (GPS) Old Navy brand a few years ago, is putting department stores on notice that he won’t let their problems devalue his company.

The CEO told investors on Tuesday that shoppers will spend money on apparel but only if it is “exciting” and that “exciting isn’t selling a generic product with more and more discounting.” Kohl’s (KSS) CFO, Wes McDonald, whose store sells Ralph Lauren merchandise, rang a similar note last month when he bemoaned an overall lack of “excitement” in fashion nowadays.”

But rather than sit back and watch the Ralph Lauren label wind up 80% off at a discount store, Larsson is planning to shorten production times to react more quickly to fashion trends and overhaul the supply chain to avoid overproduction, a problem exacerbated when department stores can’t sell his brand’s merchandise.

In the end, Ralph Lauren is not turning its back on Macy’s and its rivals. Without specifying how, Larsson said his company would work with those chains. Still, it’s easy to see why he would want to reduce his dependence on a retail channel that is shrinking with few signs of a turnaround.