FORTUNE — Let’s get this out of the way up-front: I write for Fortune, which is a competitor to Forbes. And as writers for either publication can tell you, readers mix the two up as if we were lesser Baldwin brothers at a Long Island barbecue. But my distaste for the Forbes Midas List — its annual ranking of venture capitalists, which returned yesterday — long predates my Fortune affiliation. So what follows is personal rather than professional or, perhaps more accurately, professional rather than partisan.
Ranking the world’s top 100 venture capitalists is an impossible task, so long as you’re trying to do so with quantitative rigor. After all, how do you judge investors without usually knowing how much they paid for their shares in the first place?
That’s the fundamental problem with Midas List, which focuses almost exclusively on how much investments were worth at the time of exit. Yes, earlier-stage investors are given more “credit” than are later-stage investors — the specific calculations are kept under lock and key — but specific entry prices are ignored. Even when they are known, as often is the case for companies that go public via initial public offerings.
To be clear, I’m not blaming Forbes for shoddy research here — no one else could do better. Instead, I’m criticizing them for continuing to work on, and publish, lists that it knows will be de facto specious. It’s kind of like how I don’t fault someone for failing to climb Mt. Everest on roller skates. Instead, I blame him for trying to do so in the first place.
But I do like a good list, so here are my lumps of pyrite for this year’s Midas List:
1. No demerits for large, crappy deals: Scott Sandell of New Enterprise Associates ranks #8 this year, thanks to big hits like WorkDay (WDAY) and Fusion-io (FIO). But shouldn’t he also lose points for Fisker Automotive and Bloom Energy? Or what about demerits for other VCs who held onto post-IPO shares for too long (such as KP with Zynga, or North Bridge with A123). Again, Midas List seems way too comfortable with equations that are missing key values.
2.Credits: Marc Andreessen is listed at #2, and the “big deal” he’s credited with is Skype. For sure, Skype was a great outcome for Andreessen Horowitz and its investors. But it wasn’t a venture capital deal, even by the most liberal of definitions. It was a controlled, multi-billion dollar carve-out of a mature business from a publicly-traded company. Moreover, Ben Horowitz (ranked #17) was the partner who actually sat on Skype’s board after the deal, so why is his “big deal” listed as Nicira? For what it’s worth, Horowitz tweeted that “giving either Marc or me credit for what the firm does is pretty weird given the way we work.”
3. Omissions: The University of Texas Investment Management Co. has invested in more than 100 venture capital and private equity funds, and publicly discloses the top-line performance data. Currently only two of those firms — Spark Capital and Union Square Ventures — have better numbers than Foundry Group, the Colorado-based firm whose hits have included Zynga (ZNGA) and AdMeld (acquired by Google). But no one from Foundry is able to crack the top 100? Something seems off there. Particularly if Midas continues to include valuations of…
4. Private companies: A few years back Forbes began giving credit to investors in a select number of hot private companies (i.e., Twitter), so long as that company had raised follow-on capital at a higher valuation. The problem with this strategy is that it rewards later-stage VCs who get into well-known companies at high valuations, at the expense of earlier-stage VCs who do lower-profile deals that actually generate better IRRs. For example, imagine you invest in Airbnb at a $1 billion valuation and it later raises money at $1.2 billion. That “gain” gets included in Midas List calculations. But if you invest in a different startup at a $10 million valuation and it later raises Series C funding at $300 million, Midas List ignores you.
5. Speaking of early vs. late: It doesn’t seem that the extra credit for “getting in early” always worked. Take the investors in Splunk (SPLK), which was one of last year’s top VC-backed IPOs (currently has a market cap of $4.6 billion). John Connors of Ignition Partners makes the Midas List, even though his firm didn’t invest until Splunk’s last funding round — giving it an 8.9% pre-IPO stake. David Hornik of August Capital didn’t make the list, even though his firm was Splunk’s first investor and held a 15% position at IPO.
6. Founder credit? Reid Hoffman of Greylock Partners comes in third, and it’s hard to argue with his recent investment success (although most of it came as an angel investor before joining Greylock). But Forbes says his big deal was LinkedIn (LNKD), the company Hoffman co-founded 10 years ago. Do you really get Midas List credit for investing in your own company, particularly while you’re its CEO or chairman of the board? Remember, Greylock’s own investment long predated Hoffman joining the firm — so this was less about being a venture capitalist than it was about Hoffman buttressing his own founder stock.
7. Background bias: To supplement its data analysis, Forbes as “a panel of super LPs who vet ranks.” But we have no idea who these people are or, consequently, what their biases might be. If this is supposed to be about the numbers, then leave the qualitative side out of it.
8. All about IT: Not a critique, just some facts: There are only three women on this year’s list. But they aren’t the most notable “minority.” That would be healthcare investors, of which there are only two.
To be sure, the Midas List isn’t all bad. After all, I got a column out of it. So on behalf of a Fortune writer, thanks to Forbes for some morning material…
Sign up for Dan’s daily email newsletter on deals and deal-makers: GetTermSheet.com
