It has been weeks now since Greg Smith’s blistering public resignation from Goldman Sachs (GS). But based on Wall Street’s reaction — that Smith was a disgruntled rogue, and that it could have happened to any one of us — it seems that perhaps the case’s most important lesson is being missed. And it happens to be one of the most immutable rules of business.

In fact, soft culture matters as much as hard numbers. And if your company’s culture is to mean anything, you have to hang — publicly — those in your midst who would destroy it. It’s a grim image, we know. But the fact is, creating a healthy, high-integrity organizational culture is not puppies and rainbows. And yet, for some reason, too many leaders think a company’s values can be relegated to a five-minute conversation between HR and a new employee. Or they think culture is about picking which words — do we “honor” our customers or “respect” them? — to engrave on a plaque in the lobby. What nonsense.

An organization’s culture is not about words at all. It’s about behavior — and consequences. It’s about every single individual who manages people knowing that his or her key role is that of chief values officer, with Sarbanes-Oxley-like enforcement powers to match. It’s about knowing that at every performance review, employees are evaluated for both their numbers and their values, and that only four outcomes exist.

First, for employees with good numbers and good values — onward and upward. For those with bad numbers and bad values — you’re outta here.

As for employees with good values but mediocre numbers — the stance should be, we’ll give you another chance with more coaching. Your behavior has earned you that.

World’s Most Admired Companies

Which leaves the type of employee who most commonly brings companies to their knees: the one with the great numbers and crummy values. The employee who doesn’t share ideas with co-workers, who belittles customers behind their backs, who kisses up to the hierarchy but kicks down his own people — all while bringing in the numbers.

Ninety percent of the time, managers give these people a big fat pass. “I know Jim can be a real jerk,” they say, “but I just need him until the economy stabilizes.” Or “Sure, Sally’s attitude upsets everyone, but I’ve spoken to her. I think she’s going to come around.”

Actually, all Jim and Sally are doing is sending a big fat message to every other employee: Our company’s values are a joke. And the only antidote is that Jim and Sally need to be sent home, and not with the usual “They want to spend more time with their families” BS out of the lawyers and HR, but with the truth. “Jim and Sally had great numbers,” everyone needs to be told, “but they didn’t demonstrate the values of this company.” We guarantee that such a public “diss play,” to put it more politely, will have more impact than a hundred “Our values really, really matter!” speeches by the CEO.

The Smith case occurred on Wall Street, but, to be clear, we’re talking about a problem that exists well beyond the canyons of lower Manhattan. “Values drift” is pervasive in companies of every ilk, from sea to shining sea. Employees either don’t know their organization’s values, or they know that practicing them is optional. Either way the result is vulnerability to attack from inside and out, and rightly so.

Look, it’s Management 101 to say that the best competitive weapon a company can possess is a strong culture. But the devil is in the details of execution. And if you don’t get it right, it’s the devil to pay.

This story is from the April 30, 2012 issue of Fortune