(Poets&Quants) — On November 11, 2010, the deans at Southern Methodist University’s Cox School of Business were on edge. BusinessWeek editors were online, releasing the magazine’s biennial ranking of the best full-time MBA programs in a live countdown from the lowest-ranked school that year to the highest.
As Cox administrators gathered around their computers, they watched No. 18 — Cox’s rank in 2008 — go to New York University’s Stern School of Business. A few more schools were announced, but still no SMU. Dartmouth’s Tuck School of Business came in at No. 14, followed by Cornell University’s Johnson School.
Where was Cox? Had it cracked the top 10? Finally, the school popped up at No. 12, its highest rank ever in the BusinessWeek survey. Indeed, it was the highest rank ever achieved by the school in any of the top five most influential rankings of business schools. Cox’s staff indulged in a brief celebration and drafted press releases trumpeting the school’s six-point rise.
BusinessWeek, however, was less satisfied with the results, even slightly embarrassed by them. “Everyone was saying it didn’t pass the smell test,” explained BusinessWeek associate editor Lou Lavelle in a 2011 interview with Poets&Quants. “We looked at why that was happening and the reason was we had some schools, like SMU, which had very few [MBA job] recruiter mentions — not a really deep base of recruiters — but that small base was wildly enthusiastic about the school.”
The magazine then changed its methodology to favor schools with larger, and sometimes less enthusiastic, recruiter bases. As a result, SMU’s business school slid 17 spots to No. 29 in 2012. Nothing at the school had changed: not the MBA program, not the faculty who teach there, or the quality of students. If anything, the program had gotten better, thanks to the higher rank BusinessWeek assigned the school two years earlier, which brought the school more applicants.
But now SMU’s MBA program had plunged in an unprecedented way: from 18th in 2008 to 12th in 2010 to 29th in 2012. Cox officials were flabbergasted. “We were absolutely astonished because the methodology was changed after everything was complete,” says Marci Armstrong, Cox’s associate dean of graduate programs. “While that’s certainly legal, and they certainly have the right to do it, to me it seemed incredibly unethical…. It was stunning to be really honest with you — not that we would move in the ranking, but that they would make a substantial change in the rankings because they didn’t like how we were ranked.”
Like it or not, Cox isn’t about to withdraw from the rankings rat race. “You can’t not play the game if your competitor schools are doing the same,” Armstrong points out. “If you’re a Wharton or Harvard, you can make a decision not to participate, but for the majority of schools, it’s not a choice we can make.”
Hate them or love them, B-schools ignore rankings at their peril. Applicants, alumni, and employers obsess over them. And surveys have shown that rankings are the most regularly consulted source for MBA applicants — more influential than work colleagues, family and friends, current students, alumni, faculty, or MBA admissions consultants.
These annual or biennial lists affect everything from the number and quality of applications a business school receives to the quantity and quality of employers who recruit there. They even have an influence on faculty recruitment. Srilata Zaheer, dean of the University of Minnesota’s Carlson School of Management, tells the story of a young and highly promising Chinese scholar who decided not to join the Carlson faculty because her mother implored her to take an offer from a higher ranked school–even though her discipline–management information systems–is a major strength at Carlson.
“Nearly all schools have a love-hate relationship with rankings,” concedes Idalene “Idie” Kesner, dean of Indiana University’s Kelley School. “You love them when they go in your direction, and you are frustrated when they don’t. For a long time, we said that if you do good things, the rankings will follow. We have woken up to the fact that we need to focus on the rankings in and of themselves.”
On the one hand, B-schools rely on rankings as a benchmark against their peers, a third-party guide for students, and a marketing tool. On the other, a proliferation of rankings has created widespread fatigue, a serious time and resource drain, and a potential PR disaster when numbers take a nosedive. Too many applicants take them at face value, not understanding that the results of any ranking might be based on little more than the subjective judgments of a journalist, with little, if any, understanding of business education.
In his pivotal New Yorker piece on rankings, Malcolm Gladwell recalls an anecdote in which Thomas Brennan, a former chief justice of the Michigan Supreme Court, sent out an informal survey to roughly 100 lawyers. He asked them to rank a list of 10 law schools, ranging from Harvard and Penn State to John Marshall. The attorneys positioned Penn State in the middle of the pack. The problem? Penn State didn’t have a law school at the time. “Reputational ratings are simply inferences from broad, readily observable features of an institution’s identity, such as its history, its prominence in the media, or the elegance of its architecture. They are prejudices,” Gladwell concludes.
These assumptions are perpetuated through rankings. Take, for instance, BusinessWeek’s Executive MBA ranking: EMBA program directors are asked to identify the “best” programs and rank them from 1 to 10 — this assessment amounts to 35% of the final ranking. Directors no doubt reference the rankings to assess their peers, creating a self-fulfilling prophecy.
Penny Paquette, Tuck’s assistant dean for strategic initiatives, calls it a “beauty contest.” “People vote for themselves, they have personal preferences that are not necessarily based on true knowledge of the school,” she says. “When you’re asked to pick the top 10 schools in this field out of a list of 200, how do you do that and on what basis do you do that?”
BusinessWeek began ranking MBA programs in 1988. It was a relatively simple survey. Now, given all the rankings that have followed BusinessWeek’s list, it’s a time-consuming job. Many B-schools have a dedicated professional, ranging from a media relations director to a compliance officer, who aggregates the ranking data; other schools spread the work across multiple personnel. Regardless of the person in charge, top rankings require the input of multiple offices and deans, who must review and sign off on the stats they deliver to publications.
Tuck’s Paquette spends roughly one-third of her time on rankings and supervises another employee who devotes half his time to them. The school participates in five rankings each year, and some applications take months to complete. Numbers such as employment statistics and undergraduate GPAs must be collected from departments across the school. But often the stat is only a starting place, according to Paquette.
At Cox, Martinez manages the rankings almost single-handedly. The school participates in at least eight major rankings, and juggling all the forms is no easy task. “I literally sweat bullets over these answers. I’ve become the most paranoid person that works at this school because I’m always afraid I’m the one who’s going to somehow mess up,” she says.
A proliferation of rankings has undoubtedly diluted their value overall. When 10 rankings dub 10 different schools No. 1, it’s difficult to determine who’s really top dog. Organizations ranging from Military MBA to Canada’s Corporate Knights now offer their own orderings of B-school programs. “We turn a lot down because we just statistically and emotionally can’t participate in all of them,” says Lynda Oliver, Cox’s assistant dean of marketing and communications.
Despite their drawbacks, rankings’ reach and influence mean that few business schools dare shun them. “Rankings are undoubtedly an important signaling tool, and hence, they fuel the brand of the school with prestige,” says Erik Schlie, the associate dean of MBA programs at IE Business School in Spain.
But on the flip side — a major fall in the rankings or a reporting gaffe — can cost schools big time. Rankings are a signal of quality to incoming students and potential employers, so a drop can reduce applications and deter future employers from seeking a B-school’s MBAs.
Armstrong says Cox’s slip in BusinessWeek’s full-time MBA ranking hasn’t significantly affected applications. But the school made a point to communicate the methodology change to anyone questioning the fall. She’s also adamant that while schools can’t ignore rankings, they also can’t be ruled by them. “My philosophy is that’s just one part of what you are,” she says.
Cox has certainly experienced the highs and lows of the rankings roller coaster. Last year, BusinessWeek ranked Cox’s EMBA program No. 3, right behind Northwestern Kellogg and Chicago Booth. “We were kind of waiting to get that, ‘Oh, we changed the methodology again,’ especially given that it was us,” says Oliver, an assistant dean. She recalls receiving several calls from the publication verifying their information. “I think they were like, ‘Oh God, we’ve just knocked them down under full-time and now we have to say they’re No. 3.’ That felt pretty sweet.”
More from Poets&Quants:
