In a recent interview with the Wall Street Journal, Riggio said he would step down as executive chairman after Barnes & Noble’s annual meeting in September. The 75-year-old, who also served as the company’s CEO until resigning in 2002, says he will still sit on Barnes & Noble’s board of directors after handing over the chairmanship to non-executive board member Paul Guenther.

Riggio said, simply, that he’s no longer going to be in charge of the retailer. “I’m done with being top banana,” he told the Journal. In addition to being Barnes & Noble’s executive chairman, he’s also the bookseller’s largest individual shareholder with a 17.5% stake, but says he has no plans to sell or add to his stockholdings.

The vast Barnes & Noble empire grew out of a college bookstore Riggio opened in 1965 in Manhattan’s Greenwich Village. He later bought Barnes & Noble, then a single store in New York City, and the B. Dalton Bookseller chain, eventually devising a strategy to open book superstores. He took Barnes & Noble public in 1993, and at one point in 2008 the company operated nearly 800 stores.

In his half century in the industry, Riggio has witnessed the tremendous rise of the mega-bookstore—whose aisles and in-house cafes became a destination for browsing patrons—and then the concept’s precipitous decline. Barnes & Noble’s big-box bookstores were once considered a killer of independent booksellers, but the company ran into its own buzzsaw in Amazon (AMZN). The company has tried to compete with its own e-commerce strategy and Nook e-reader, but has largely failed to fend off Amazon, whose cheap prices and popular Kindle device have lured away customers at an incredible clip.

Since 2008, Barnes & Noble has closed about 150 stores and now operates 640.

In its most recent quarter, Barnes & Noble’s total sales fell 1.8% to $1.4 billion. Its online sales plunged 12.5% due, in part, to a botched website relaunch. That dip was offset—somewhat surprisingly—by sales at the retailer’s brick-and-mortar stores, which rose 1.3% in the third fiscal quarter. The company said it would shutter just eight stores in the fiscal year, the smallest number of closings since 2000.

Industrywide, sales of physical books have managed to hold their own against e-books. That might be seen as an indication that the company Riggio built from scratch can hold off Amazon’s seemingly fatal assault, but—in perhaps the ultimate insult to legacy bookstores—Amazon recently introduced its own physical store strategy. The company opened one store location in Seattle in November and is said to have plans for hundreds more.