David Geffen may be the most versatile American entrepreneur of the last three decades other than Steve Jobs, though Jobs himself sometimes called him for counsel.

Peerless in his ability to merge artistic and financial instincts, Geffen has been an agent, a manager, a record-industry mogul, a movie and Broadway producer, and a co-founder of a film studio. Many individuals, at different times, have been called “the most powerful man in Hollywood” — Geffen was for a generation. He’s been a confidant of high-ranking Democrats, as well as a philanthropist for medicine, the arts, and AIDS research. He was a millionaire before he was 30 and a billionaire by 50. He’s amassed a fortune of more than $6 billion, the largest part of which is a nonpareil collection of postwar American paintings. He’s given $300 million to the UCLA medical school. His friends range from Silicon Valley (Larry Ellison and Larry Page) and entertainment (Steven Spielberg and Barry Diller) to journalism (Maureen Dowd) and fashion (Calvin Klein). In the music business, where he first made his name, his clients have included John Lennon, Bob Dylan, Elton John, Donna Summer, the Eagles, Joni Mitchell, Guns N’ Roses, and Nirvana.

Born into a working-class family in Brooklyn, he dropped out of college and was driftless until he discovered the world of agency — an entertainment niche that relied not on specialized knowledge but guile and relationships. Geffen combined those skills with an eye for talent, be it the voice of Jackson Browne or the onscreen charisma of Tom Cruise. His corporate creations have included Asylum Records in 1971 (which he later sold to Warner Communications); Geffen Records in 1980 (which he sold to MCA for $550 million); and DreamWorks SKG (of which he’s the “G”) in 1994.

Now 70 and retired, Geffen rarely gives interviews. But in a series of conversations with Fortune’s David A. Kaplan, he reflected on a life in entrepreneurship — what he called “an incredible ride.” In his splendid apartment overlooking Central Park in Manhattan — and with a view of his Jasper Johns and Roy Lichtenstein treasures inside — Geffen talked about fortuity and fame, allies and adversaries, as well as Apple, Hillary, yachts, and what it’s like to have a celebrated song written about you. Edited excerpts:

Q: You started becoming David Geffen 50 years ago.

Geffen: My first job in the entertainment business was at 19 — as an usher for CBS in Hollywood. All of a sudden I see The Judy Garland Show being taped there. And I thought, “Wow! Show business, this is good.”

Did you have an itch to be in show biz?

I was a dumb kid from Brooklyn, but I was fascinated with it. I had read books about it when I was a kid, like the biography of Louis B. Mayer Hollywood Rajah.

You were in L.A., but then returned to New York?

I moved to L.A. the day I graduated high school. I just wanted to get away from Brooklyn and my parents. But I was fired at CBS the day J.F.K. was killed. Somebody said something nasty about him, and I got into a fight. I was broke.

How did you wind up in the mailroom at the William Morris Agency?

I was a receptionist at Richelieu Productions and asked a casting director how to get more involved in the entertainment business. She said, “Do you have any talent?” I said, “Absolutely not.” She said, “Have you got any gifts whatsoever?” I said, “I’m giftless.” She said, “Maybe you should become an agent.” I said, “What do you have to know?” And she said, “You don’t have to know anything!”

So?

I got an interview at the Ashley-Famous Agency and filled out the application honestly. And Al Ashley says to me, “You should’ve lied! Who would hire you with this résumé?” He didn’t. So I looked in the Yellow Pages under “Theatrical Agencies” and the biggest ad was for William Morris. I said I graduated from UCLA. They hired me immediately.

And then?

The guy who showed me around on my first day got fired. He had lied on his application. And I thought, “Oh, shit!” So I came in early every day for six months till I intercepted the letter from UCLA saying they’d never heard of me. I replaced it with a letter that said I’d graduated.

Do you ever get asked in some ethics class, “That’s a charming story of self-invention, but would you counsel somebody else to lie on an application?”

Look, I’m not setting an example. One doesn’t have to counsel anybody to lie. People lie. But it’s an idiotic thing that you have to be a college graduate to be an agent. In any case, did I have a problem with lying to get the job? None whatsoever.

LUCK AND AMBITION

Why did you thrive?

I wasn’t ambitious at 20 because I didn’t think I could do anything. But the epiphany for me at William Morris was realizing, “They bullshit on the phone! I can bullshit on the phone too!” It wasn’t so much that you were talented — it’s that you didn’t particularly need to be.

How long did it take you to have that epiphany?

A week.

How long did it take to stand out as an agent?

After four years, I was offered a lot of money to go to Ashley-Famous, which had rejected me in the first place. I told Ted Ashley, “I’ll come if you pay me $1,000 a week and give me the office next door to you.”

Was that a good move?

I was resented immediately — for having that office and because people had heard I was being paid $1,000 a week at the time I was 24.

What had you been earning?

Four hundred dollars. I eventually concluded I could make more money as a manager than as an agent. All you needed was a star. I had signed a singer named Laura Nyro, and I thought she could be one. I became her manager, and we were partners in her music publishing company. That was 1968. Less than two years later, I sold that company to Bill Paley [the CBS magnate] for $4 million. And I was a millionaire by the time I was 29.

Do you ever reflect on this transformation?

No. I never had a plan for anything. I was just putting one foot in front of the other and trying not to make a fool of myself.

Whom did you like hanging out with more — the musicians or the moguls?

Different hangouts. They’re all interesting. I enjoy being with Lloyd Blankfein [of Goldman Sachs]. I enjoy being with Joni Mitchell. I’ve enjoyed my time in the Middle East: Sheikh Abdullah [of Abu Dhabi] said I was the only American they ever met who didn’t want any money from them.

You’re the person Mitchell wrote about in “Free Man in Paris” in 1973 — about the pressures of show biz. What’s it like to be immortalized?

I tried to get her not to record it! It was so revealing that I was embarrassed.

Could you tell it was about you — “I deal in dreamers and telephone screamers”?

I instantly knew it was about me. I’m proud she wrote it. It’s a wonderful song, and it’s accurate. Joni’s a very perceptive girl.

What about being mentioned a few years ago as the possible inspiration for Carly Simon’s “You’re So Vain”?

I laughed. It’s clearly not about me. It was about people unlike me: Warren Beatty and Mick Jagger and others who are speculated about.

SILICON VALLEY AND WEALTH

Your entrepreneurial mentality makes me think you’d have an affinity for Silicon Valley.

I don’t. Remember, I couldn’t have done anything in Silicon Valley. I’m not smart enough.

Do you ever visit there to see new things?

I’m not one of these guys who are looking to get in on the IPO of anything. My interest is as a consumer, as an admirer of people of great accomplishment who have made products I admire.

Do Silicon Valley types come to you?

I certainly have enjoyed my relationship with Sergey [Brin] and Larry [Page] and Larry Ellison, Steve Jobs, Paul Allen, and others. I’ve learned a lot from them and given them a lot in return — but not in the areas they’re geniuses at.

For example?

Art. Or show business, in Jobs’ case.

So, for Larry [Page] and Sergey, you might spark an interest in paintings?

Well, so far, not. But Larry did ask for advice about boats — which he didn’t take, by the way. [Laughs.]

What was the advice?

Get a better boat.

Do you look at the list of richest Americans when it comes out?

Yes. Do I care at this point in my life? No. But the first time you get on it, it brings a smile to your face. I think people are flattered at the beginning and annoyed at the end.

You knew Steve Jobs well. I wonder what he thought of being on that list.

I don’t think he cared very much about money. Steve was a guy who did not think it was a good thing for his family. He said that to me many times. He didn’t like demonstrations of wealth. He used to make fun of my boat.

Did you talk to Jobs a lot?

He was someone who called for advice — like when he bought Pixar and decided to make Toy Story.

Why do you suppose you and Jobs didn’t have any difficult moments?

Because our relationship was based almost always on him calling up and asking, “Is this a good deal?” I’m not in his business, and I wasn’t trying to be. I was just a guy who admired him a lot and was flattered he wanted my advice.

Will Apple endure without Jobs?

Steve was dazzling. He wasn’t an inventor; he was a marketer. And a genius editor — in design and function. And, God knows, he’ll be missed. But I believed in the company when he was alive, and I believe in the company now that he’s gone.

If Ellison and Jobs represented opposite ends of a spectrum on the value of wealth, where would you fit in?

Jobs and I are not similar in that regard. He didn’t give a shit about money, and I do give a shit about money. I wanted to have a lot of money. I thought it would change my life, which it did, but not in the ways I expected.

Did you join the Giving Pledge of Warren Buffett and Bill Gates?

No, because I don’t believe in it. It’s grandstanding: You’re just saying you’re going to do it. There’s no legal obligation. I know more than one member of the club who’s told me, “Well, you’re not actually required to give anything away, but you look good.”

Do you get pressure to join?

Yes. From Gates. At his house when he invited me to come up to talk about it recently.

What will you do with your wealth?

I intend to give every nickel away. I’m just not joining some club of rich people in which to do so. Actions speak louder than words. If you’re going to give it away, then give it away. And do it when you’re alive.

You have strong views on the atmospherics of philanthropy. Why not give away money anonymously, instead, say, of having the UCLA medical school named after you?

I don’t agree that the best giving is anonymous. We should be examples to our friends and communities. I should be an example to young, gay kids. And it’s not like people are really anonymous anyway — you remember that episode of Curb Your Enthusiasm? In the end, everybody knows who “anonymous” is.

INVESTING AND ART

You strike me as someone who’s fundamentally a builder.

I’m an entrepreneur. I couldn’t have maintained a job anywhere, so my only chance was to start my own businesses.

Did you worry about failure?

I have to be very honest and say I never thought I would fail.

You had that short-lived position as vice chairman at Warner Bros.

I was fired after nine months. I did have moments of failure in my life, but at the end of the day, no. Record companies, Broadway, DreamWorks — all successful. But let’s be clear: I made most of my money investing the money I made from selling my companies.

What was so special about your investments?

There have been pivotal moments that turned out to be extraordinarily smart. For instance, in 1991 through 1994, I spent $400 million buying the greatest post-World War II paintings in the world. In 1993, I gave $200 million to [hedge fund manager] Eddie Lampert. At that point I had about 60% of his fund, and people thought I was insane. He compounded it at 30% for a long time.

How much of your net worth is in the market?

I have almost a billion dollars in Apple alone. I have a considerable amount of money in the stock market, and I have a considerable amount in paintings.

If Kleiner Perkins asked, “You want to be a limited partner for a million?”, you’re not interested?

No. I like to be able to change my mind — to access my money when I want.

Your art collection now is worth?

A couple of billion dollars.

That should put you in the pantheon, shouldn’t it?

I think I’ve sold at least three or four of the most expensive paintings ever — a Jasper Johns, a de Kooning, a Jackson Pollock.

What made you get interested in art?

I used to brown-bag it at the Museum of Modern Art when I was in the mailroom at William Morris. I used to go with my friends, and we’d walk around and say, “Can you believe that painting is $5,000?” It became an interest from that time on.

You didn’t see art as an investment vehicle?

No. When I became wealthy enough to buy the kinds of paintings I’d love to hang, I bought them.

But then you sold some in 2007.

Yes. I believed a crash was imminent and there would be great opportunities like there had been during the S&L crisis in the early 1990s. I had sold [Geffen Records], the crisis happened, and in less than two years I doubled my money by buying up bonds — RJR debt, Warner, others. I thought this was going to create the same kind of opportunity. But I never bought anything. So I regret selling the paintings.

And art prices themselves didn’t collapse in the meltdown?

They went up, in fact. It’s better than gold. You can’t put gold on your wall.

L.A., N.Y., RETIREMENT

You’re connected in three different cultures: Hollywood, Wall Street, and Silicon Valley. Do you have a preference?

They’re all interesting in different ways. The other night, I went to the costume ball at the Met. The Academy Awards are not nearly as glamorous as what Anna Wintour puts on for the costume ball.

Who’d you go as?

Myself — David Geffen. I buy my clothes at J.Crew online. I’m a very casual guy.

How do you divide your time?

One-third in New York, one-third in L.A., and one-third on the boat [the 453-foot yacht Rising Sun he bought from Ellison].

How does a kid from Brooklyn get interested in yachts?

In 1974 I was invited onto Sam Spiegel’s boat, Malahne. It was 180 feet — gigantic back then. He had acquired it during Lawrence of Arabia. And when I arrived, there was [Princess] Grace Kelly, [publisher] George Weidenfeld, [conductor] Herbert von Karajan, [actress] Romy Schneider, and Jack Nicholson. We went from Monte Carlo to Portofino. It was thrilling.

Are you surprised you don’t seem to miss what you did for so long?

I made a decision five years ago I didn’t want to work any longer in the entertainment business. I didn’t love the music or the movies that were being made, and I don’t think you can be successful if you don’t love it any longer.

Do you like music and movies less because of the product — or because you’ve changed?

All I can say is when I go to see Iron Man 3, I’m not as enthralled as I was when I saw The Bridge on the River Kwai.

Music too?

I’d rather listen to Ella Fitzgerald sing the songbooks. Rodgers and Hart, Cole Porter, Duke Ellington, Harold Arlen. I can listen to the music of my time until pretty much the beginning of the ’90s, until Tupac.

Do people still call you?

If my friends want any assistance, I give it. I do it just for the fun of using your brain. When I say I don’t work, I mean I don’t have any obligations. My day is still full — between taking care of my own portfolio and being involved in other people’s stuff.

Do you wish you’d stopped working earlier?

I couldn’t. I had a responsibility to DreamWorks. The day [in 2008] we made the deal for Steven [Spielberg] and Stacey Snider to move DreamWorks to Disney, I retired. I wanted to be sure they were in good hands. Bob Iger is a sensational executive, and Disney is an incredible company.

Are you glad you did DreamWorks?

I joined it to help my friend Jeffrey [Katzenberg] out — he had been fired at Disney. I didn’t realize I was making a commitment that was going to last so many years. Believe me, if I had thought it through, I wouldn’t have done it. But I raised $1 billion in equity for DreamWorks and $1 billion in debt. I promised Paul Allen, Jimmy Lee at JP Morgan, and the other investors they would get their money back. And they did.

You’ve had famous feuds over the years.

I aspired to be a good friend, and I am. There are people who would tell you that I’m a very bad enemy to have. What they’re really saying is if people want to fuck with me, I’m up to the task.

Where does that come from?

We all have a line we don’t want crossed. And if you’re a young, small, thin, gay guy — when people try to push you around — I never made it easy. When I was in junior high school, I used to hide my lunch money because these big kids would take it, and I got beat up. My brother said to me, “You know, you’re going to get beaten up one way or the other. You’d better fight back.” So I learned early: I don’t want to have a fight, but there has to be a cost in fighting with this guy.

You weighed in during the 2008 presidential campaign — siding with Barack Obama and criticizing the Clintons. Any second thoughts?

I’d heard Obama speak at the 2004 convention and I called him up and said, “You’re going to end up running for President, and I’m going to support you.” He laughed. Several years later, he called and asked, “Remember when …? Can I count on you now?”

Would you support Hillary if she runs in 2016?

Absolutely.

Any thoughts on what your epitaph should be?

I’m not going to have an epitaph. I’m going to be cremated. Because the whole idea of legacy is such bullshit.

You have a legacy, no?

I don’t think so. Bill Paley was a giant of the first half of the 20th century. Today few remember him. And I’m not Bill Paley. That’s the way of the world.

This story is from the August 12, 2013 issue of Fortune.