Chenault used the New York Times DealBook conference Thursday to speak out about the lack of diversity in the top ranks of the biggest companies. And he insisted that it’s because of management – not because of lack of talent among minority candidates.

“The fact that we’re in this situation is a real problem and embarrassing for corporate America,” Chenault said.

To make matters worse, barring major executive shakeups, the number of black Fortune 500 CEOs will fall to just three when Chenault, 66, retires in February. Chenault’s replacement at American Express, Stephen Squeri, is white.

Black CEOs account for less than 1% of Fortune 500 companies, but African-Americans make up more than 13% of the U.S. population.

The biggest reason for the lack of diversity, Chenault said, is the lack of initiative from upper management. Managers should be actively selecting women and minority candidates for jobs that can push them to the top, he said.

“You’ve got to put them in the pipeline… to identify, recruit, develop, and push people forward,” he said Thursday.

Chenault added that a major problem driving the racial divide: the lack of frank discussion about the issue.

“Someone said to me ‘When I look at you Ken, I don’t see color,’ ” Chenault recollected.

His response?

“Then you’ve got a problem.”