He came to Apple (AAPL) in 2001 with plans for building what would become the iPod. By 2006 he had replaced Jon Rubinstein — who went on to build the Palm (PALM) Pre — as head of Apple’s iPod division, in charge of both what was then the company’s biggest cash cow and the project that would become the iPhone.

Two years later, he and his wife, Danielle Lambert, vice president of Human Resources, were out — “reducing their roles” within Apple, according to the press release announcing his successor, to “spend more time with their young family.” He remained as an adviser to Steve Jobs, but had no visible role within the company.

At one point, outsiders thought Fadell might have been No. 2 in the line of succession at Apple, after COO Tim Cook. This is what we wrote about him five months before he was replaced:

Tony Fadell
Title: Senior vice president, iPod division

With his American swagger and his hair bleached white, Fadell stood out at button-down Philips Electronics, where he led an in-house pirate operation designing Windows CE-based devices. It was there that he came up with the idea of marrying a Napster-like music store with a hard drive-based MP3 player. He shopped the concept around the Valley before Apple’s Jon Rubinstein snapped it up and put Fadell in charge of the engineering team that built the first iPod. Ambitious and charismatic (and no longer a bleached blond), he now runs the hardware division that makes two of Apple’s three key product lines: the iPod and the iPhone. (link)

His final departure was reported Monday evening in the New York Times‘ Bits column. Fadell told the Times‘ Brad Cook he was moving on to advise companies and pursue private investments with a focus on green technology.

“My primary focus will be helping the environment by working with consumer green-tech companies,” he said. “I’m determined to tell my kids and grandkids amazing stories beyond my iPod and iPhone ones.”

See also:

• Losing Tony Fadell: The man who made the iPod
• After Steve Jobs: Handicapping Apple’s back bench

[Follow Philip Elmer-DeWitt on Twitter @philiped]