Exclusive: George Zimmer on being fired by Men’s Wearhouse, and what’s next

December 9, 2013, 3:00 PM UTC
George Zimmer

FORTUNE — It was the firing heard around America. George Zimmer was out at The Men’s Wearhouse (MW), after 40 years and countless television ads in which he guaranteed that customers would like the way they look.

The company’s initial termination announcement was terse and without explanation, sparking social media outrage among some customers and fans. It then provided a more detailed explanation, essentially alleging that Zimmer was unable to accept that he was no longer CEO of the company he founded — Zimmer handed the reins over to Doug Ewert in 2011, but remained executive chairman — and alleged that he had been working to take the company private. Zimmer hired a PR firm and responded in a blistering public letter.

Since then, The Men’s Wearhouse has continued to be in the news. First for buying Joseph Abboud, and then when smaller rival Jos. A. Bank (JOSB) offered in early October to acquire the company for $2.3 billion. That offer was rebuffed, with The Men’s Wearhouse then offering to buy Jos. A. Bank for $1.54 billion, in a so-called Pac-Man defense.

But Zimmer went silent. He took that PR firm off retainer and has not made any public comments about his firing or future since June. Until now.

What follows is an edited transcript of a phone conversation, held with Zimmer last Thursday. In it, he discusses his termination, his thoughts on a Jos. A. Bank merger, and why he seems more popular now than ever before:

FORTUNE: How did you learn that The Men’s Wearhouse was planning to fire you as executive chairman?

ZIMMER: I got an email about 10 days earlier, which was extremely harsh and mean-spirited. It said that my job status and compensation would remain the same “at this time.” Then they basically began throwing me out of my office, and that was as traumatic as when I was actually terminated because, at that point, it was pretty clear what was happening.

We had a scheduled board meeting on a Tuesday followed by a scheduled shareholder meeting on the Wednesday, so I checked into the hotel on Monday evening. A couple of directors came to see me individually to explain what was happening the next day. I wasn’t being summarily discharged, but instead they offered me a figurehead position, with fair compensation. But I elected not to work for people who I had originally hired to work at the company, who now just wanted me in that sort of role. So I declined.

I was going to be reelected director at the annual shareholder meeting the next day, because the proxies had already been out for weeks. So they then said that they wanted me to resign as director as well as be terminated as an employee, and if not they’d postpone the shareholder meeting, which is what they did. Then they sent out new proxies in which I wasn’t nominated …

When I was terminated I got no severance. They even took my phone and cut off my insurance.

MORE: 20 top picks from 20 star investors

Had they gone through with the previously scheduled shareholder meeting, would you have stayed on as a director?

I would have probably stayed on, although in the exact moment I felt like “I don’t want to work with you any more than you apparently want to work with me.”

In explaining your termination, the company claimed that you had been pushing to sell The Men’s Wearhouse to a private investment group – something the board disagreed with. True?

I didn’t have any conversations with private equity firms. I spoke with [investment bankers] — people who evaluate if a going-private makes sense. We had been approached over the last five years about going private on a number of different occasions, and our response, usually initiated by me, was absolutely not.

But I was at a San Francisco conference put on by Whole Foods CEO John Mackey, called Conscious Capitalism — at which I normally present. And I was approached by some people who asked why we weren’t considering going private. I gave the standard response about not being interested, at which point I was told that we could go private, and our shareholders would get a 30% to 40% premium and that interest rates were at a historical low point. All of a sudden a light bulb went off in my head, and I spoke to these [bankers] who confirmed that this looked like a good idea.

So I got the board together telephonically and explained what I thought we ought to do: Invite the [banker] to a board meeting to explain to all of us what’s involved. But I now believe that, prior to my call, they’d heard enough rumblings, retained counsel, and declared me persona non grata. So my idea wasn’t going anywhere. At least that’s my perception.

The company also argued that you “refused to support the team unless they acquiesced to his demands.” Legitimate criticism?

There is some truth to the idea that, after the first year of my successor’s administration, I began to lose confidence in him. But it’s not like all of a sudden I went from loving him to not liking him. It was a process that went on over months. At no point did I overrule his decisions. The way the board has framed it is somewhat ironic, because I swallowed my own instincts and allowed the company to go down the path he was embarking on.

At the end of last year we had a board meeting where I first spoke privately to the board, followed by Doug speaking privately to them. I did indicate that Doug and I were having some disagreements, but I don’t know why anyone would think I was giving them an ultimatum. At this point in time, I don’t think The Men’s Wearhouse is the organization that I had worked to create over 40 years.

MORE: The worst (and best) company name changes

What’s different?

Let’s start with the fact that I’m not on TV saying “You’re going to like the way you look. I guarantee it.” Hundreds of people, customers, have stopped me on the street to ask what happened. It’s a very awkward situation for me, because they felt a connection. Maybe it’s not unlike how many of us felt a connection to Mr. Whipple or other advertising figureheads. So the simple answer is that the company has a brand problem that will get worse, not better.

It also seems to me that the company’s culture is different, but the truth is that it’s been drifting for several years.

After your termination, there was speculation that you wanted to buy the company back. Did you have those conversations?

Back in June there were a number of large private equity companies that approached me about exactly that. My answer was that I’d certainly consider it. As time went on, though — and particularly since Jos. A. Bank got involved a couple of months ago — the price got … not overpriced, but fully valued.

So to come to the founder of a business and ask him to pay up for having your old job back after six to 12 months of the company going in a direction I don’t agree with, in the end I just couldn’t bring myself to say yes. Even though I’ve talked to private equity people about other business opportunities, I’m not really thinking about being involved in The Men’s Wearhouse anymore.

Jos. A. Bank offered to buy The Men’s Wearhouse in October. The Men’s Wearhouse said no and then turned around to offered to buy Jos. A Bank. What was your reaction when all of this began?

I was initially surprised, but then I began to think it made a lot of sense. The Men’s Wearhouse had talked about buying Jos. A. Bank 10 years ago and then again a couple of years ago. For many reasons it makes a lot of sense for Jos. A. Bank to buy The Men’s Wearhouse, but also for The Men’s Wearhouse to buy Jos. A. Bank. Neither company has any debt, and it’s never been a better time in terms of interest rates.

The beauty of my current position is that I don’t really care, except that I have around 1 million shares of The Men’s Wearhouse stock. If forced to make a bet, I’d say 2-to-1 that a deal is going to get done.

MORE: How to bet against the bitcoin megabubble

Did Jos. A. Bank reach out about a role with the merged company, were they to win?

No. In all these years I never had a relationship with the senior execs at Jos. A. Bank. I always saw them as our competitor, so I was reluctant to talk too much for fear that I may say something that I shouldn’t.

It is not uncommon for a founder to be booted from his company, but your termination seemed to strike a sharp nerve in popular culture. Why do you think that was?

It would probably have something to do with America’s love of the underdog. Clearly this appears to be a problematic situation. I think that having been an entrepreneur for 40 years in the schmatta business — having made my money not in technology or in real estate or in the financial sector, but in more of an old-fashioned way. I think that has some attraction and, over time, I did most things the right way. And people appreciate that, even if they never bought a suit at The Men’s Wearhouse.

Were you surprised to be parodied last month on Saturday Night Live?

Not as surprised as you might think. I’m not really that humble. And I didn’t think it was that funny a bit. Others have done it well, though. Jimmy Fallon’s writers, for example, have always made humor out of Men’s Wearhouse and me in a way I’ve always been happy with.

Do you want to start or run another company?

I’m almost 100% certain that I’m not going to be a public company CEO. I just turned 65, I’m on Medicare and have two kids in middle school. I’ve proven what I need to in business. The only draw for me is the store employees. The Men’s Wearhouse has 50 black-tie holiday parties in November and December every year, and I went to around 15 of them. At some level that was the glue that helped hold all of the disparate stores and employees together. This is the first time in 40 years I haven’t done it, and I do miss it.

Could you see yourself in some sort of board or advisory roles?

Absolutely. I love the idea of having more of a behind-the-scenes role. I have a wealth of experience that I wasn’t necessarily as aware of when I was running the company. That’s one reason why I’m actually writing a book — to express some of the ideas there isn’t space to do here.

[Clarification: In an earlier version of this post, Zimmer said that he had spoken with “sponsors” about taking Men’s Wearhouse private. He meant to say “investment bankers,” and the post has been updated to reflect that]

Sign up for Dan Primack’s daily email newsletter on deals and deal-makers: GetTermSheet.com