Advertisers are scaling back spending on Internet portals like Yahoo in favor of search engines and entertainment sites, according to an annual report from Microsoft’s online ad agency, Avenue A/Razorfish.

That’s bad news for Yahoo (YHOO) as one of CEO Jerry Yang’s turnaround strategies is to transform the site into a starting point for the web.

Advertising buys made by Avenue A/Razorfish on portals fell from 24% in 2006 to 19% last year, according to Jeff Lanctot, the agency’s senior vice president of media.

“We don’t think it’s a portal-dominated landscape like it once was. There’s too many places for advertisers to put their budgets,” Lanctot says. Avenue A/Razorfish spent $735 million last year, an increase of 36% from 2006. A third of the budget went to paid search and the entertainment spending grew by more than 50%. The interactive agency doubled the number of websites where it placed ads to 1,800 last year.

More choices for advertisers makes it difficult for big publishers to raise their rates substantially. That’s why both Microsoft (MSFT) and Yahoo went on a buying spree last year to snap up ad networks, where the ad dollars are flowing. Microsoft got DrivePM through its $6.1 billion aQuantive acquisition while Yahoo bought RightMedia and Blue Lithium.

No surprise that Lanctot says a Microhoo deal could make the company a digital media powerhouse by combining its ad networks. Avenue A/Razorfish spent 71% of its billings on the top five ad networks — Google’s AdSense, Advertising.com, ValueClick (VCLK), Tribal Fusion, and DrivePM. “The ad network is one that benefits greatly from scale” he says. “The more ad placements you have, the more likely you’ll be able to command higher rates. For Yahoo and Microsoft, there’s no downside to combine their assets.”

Both Yahoo and Microsoft have made modest strides to catch up with Google (GOOG), the dominant leader in paid search. Yahoo rolled out its long-awaited Panama search technology last year, and Microsoft has its new AdCenter. But neither effort is enough, which is why Redmond is eager to buy Yahoo.

“In search, it would be presumptuous to assume that Microsoft and Yahoo could catch Google if they combined forces,” Lanctot says. “But would there be a legitimate No. 2 in the market if these two combined their efforts? Maybe. As Google continues to extend its lead, it’s reasonable to think that a more combined identity rather than two separate ones would be beneficial.”