By Julia Ioffe, contributor

Hacking used to be so quaint. In the old days (the early 90s) the villains typically were attention-seeking computer geeks infecting computers with viruses that were a headache for consumers and tech departments to debug.

Today’s cybercriminals are out to inflict real harm: They can be commercial entities breaking into competitors’ records, or international crime rings stealing valuable data like credit card numbers and email passwords.

And because such cyber-heists extremely lucrative – some estimates put the size of this underground economy at $1 trillion –more players are getting into the game, developing increasingly sophisticated ways to crack into computer systems and exploit their ill-gotten gains. Viruses alone can take trillions of forms, and spam, the most popular way of infiltrating computers, accounts for some 90% of all e-mail traffic.

All of which makes it harder for computer security companies to stay one step ahead of these evolving threats. “Clearly, it’s an arms race,” says Enrique Salem, CEO of Symantec, (SYMC) the world’s largest software security company. “They’re always trying to find ways of getting around our technology, so we’ve got to keep innovating” – and getting inside the criminal mind.

Symantec, based in Cupertino, Calif., continues to deploy a set of tried and true tools to keep digital risks at bay: Last year the company generated 1.6 million automated signatures –signatures are virus-specific cures– to block known attacks. Its software also automatically blacklists and filters bad programs and sites. And the company applies advanced behavioral technology to monitor and shut down malicious software just before it’s about to do something really harmful, thereby minimizing the impact on a corporate computer system or even an individual user.

But even this aggressive, multi-pronged approach isn’t enough to stop the bad guys. Blacklists are not fast enough to catch brand-new malware; “white lists” of safe software are too restrictive. And cybercriminals now generate malware automatically so that every visitor to, say, a bad website gets a slightly different version of the bug, making individualized cures highly impractical, if not impossible.

“Most of it is generated by virus-generating software,” says Steve Trilling, a former stand-up comedian and software engineer who runs Symantec’s STAR team, short for Security Technology and Response. “There are now many tens of millions of viruses out there, and you just can’t keep scaling at that rate.”

New protection codenamed “Mr. Clean”

And so last week Symantec launched the latest version of its Norton products with yet another layer of protection called Quorum (known internally as “Mr. Clean”). Quorum works in much the same way that the Zagat’s restaurant guide does, by relying on reputation. If you want to download a program that very few people in the world have, Quorum will recommend you stay away from it but leaves the ultimate choice to the consumer. After all, the program could be a randomly generated virus – or a highly-customized piece of software.

To prevent the program from blocking good software (what’s known as false positives), Quorum checks in with the back end and, if a program checks out, Quorum will not block it and slow the user down.

Symantec is able to calculate reputation with such confidence because, for the past year, 29 million Symantec customers have been using a Quorum prototype and automatically relaying data to the Symantec mother ship, where it is anonymized and crunched.

This provides Symantec with a large database from which to compute a program’s standing – and, with nearly 60 million Symantec customers around the world, that database is going to grow at a fast clip once the software is released on a wider market. And because the calculation is fully automated and based on a massive data base, hackers will have a difficult time distorting the real number of people who have downloaded their software.

This program also takes up less space and so can be run on mobile devices, which have yet to come under extensive attack. (Though the prospect is increasingly likely, industry watchers say, the mobile-device market is still too fragmented to be profitable for security companies; nor do people make many financial transactions on their phones – yet- making cell phones and BlackBerrys less likely to be attacked.)

Thwarting the Cult of the Dead Cow

But even cutting-edge software and a massive global infrastructure staffed by 17,500 employees cannot stop every single threat. To cut down on future breaches Symantec tries to educate school kids on smart web-browsing techniques. And it works with Congress and international governments to create a uniform legal standard to bring cybercriminals to justice. (The famous case of the ILOVEYOU Bug, in 2000, illustrates the need. When Symantec brought forward information pinpointing the Filipino hackers behind the globally infectious virus, all charges were dropped because the Philippines have no laws banning cybercrime.)

But as the cybercrooks get ever smarter, Symantec also is devoting more resources to the digital equivalent of “black ops” – folks who spend their days attending hacker events and trolling the ‘net for secretive chat rooms where the bad guys boast of their conquests and tactics. Every summer, for instance, hackers gather in Las Vegas for the Defcon Conferences – and Symantec goes, too.

One year, as a hacking group named Cult of the Dead Cow presented their new hacking techniques by lobbing informational discs (and hunks of raw meat) into the audience, Symantec reps ran them back to the hotel where a team of Symantec programmers sat churning out signatures, hobbling the tactics almost as soon as they were introduced.

It may sound a bit surreal, but CEO Salem tries to put the war on computer crimes into perspective: “You’re never going to eliminate crime,” he says. “You’re never going to eliminate cybercriminals and that’s going to be an ongoing challenge.” But to paraphrase an old saw: you have to think like a cyber criminal to catch a cybercriminal.