FORTUNE — Since Microsoft released the Bing search engine in 2009, it has struggled to compete with Google. Most reviewers, and more importantly, most users, have consistently said that on average Google delivers superior results. While Bing’s market share in the U.S. has nearly doubled to 15.4%, according to comScore, the gains have largely been at the expense of Yahoo (YHOO). Google’s own share has risen by about 3 percentage points to 66.7%. Now following a new set of technical improvements, Microsoft (MSFT) is trying, once again, to change perceptions and break what it calls the “Google habit.” On Thursday, it announced a “Bing It On” challenge that will encourage users to conduct blind comparison tests between the two search engines after research by an independent firm showed that users on average prefer Bing.
Mike Nichols, corporate vice president and chief marketing officer for Bing, spoke with Fortune about Bing’s changes and the ad campaign promoting the challenge. Below is an edited transcript of the conversation.
You’re claiming that Bing is now delivering better results than Google (GOOG). What makes you say that?
When we do a blind comparison test on the results of the core quality between Bing and Google, studies done by a third party have basically been showing that people are preferring Bing’s results over Google’s nearly two-to-one.
Two-to-one, that’s remarkable. What do you think has changed?
The engineers here have been working for a while now on really applying advanced machine-learning techniques to provide better search results. And there was a partnership between the Bing team here, and the Microsoft Research team that resulted in a technology that allows for quick experimentation of different machine learning models. The product basically has been getting smarter and smarter over time.
MORE: Can this smartphone save Nokia?
Can you talk a little bit about how the independent research was conducted and are you confident that it will hold up, because it could be quite an embarrassment if we all start trying it and you don’t deliver on the promise?
[In the test] a user would have to submit a query of their choosing and then what we would do is remove the brand from the experience. And then just ask the user, do you like the results on the left, or do you like the results on the right? And of course we switch which things around. Across the range of different types of things that people search for, they tend to prefer the results from Bing.
So, now you’re asking people to do this test essentially themselves, to conduct a “Bing It On” challenge. How does it work?
You can submit any query that you like and after five queries it will say, hey, either Bing won or Google won or it was a draw, and you can choose to share your results with your friends either way if you like.
You’re doing a national ad campaign to promote this. Where will I see the ads and how much are you spending on it?
We don’t really announce the size of the marketing campaign, but it will be a nationwide campaign. We’ll launch some ads ‑‑ some TV ads to start at the MTV Video Music Awards, but you’ll see them in other places, as well. You’ll also see a lot of marketing online and even in Microsoft stores, as well.
MORE: Why you should be excited about 5G wireless
In the past, advertising campaigns for search engines have not moved market share numbers very much. What do you think may be different this time?
I think Bing is the only search engine that’s gained any appreciable amount of share over the last few years. With this particular campaign what we’re really trying to do is to make sure that people have the facts about the quality of search results. They can conclude it for themselves.
How high do you think Bing’s market share can grow in, say, a year?
We don’t project specific sort of query share numbers. But, really what we’re trying to do here is encouraging people to compare Bing and Google head-to-head, and we believe that as people do that increasingly they will see that Bing is worth breaking sort of that Google habit for, because of the quality of the search results, because it’s the only search service that includes useful information from your friends and experts from Facebook, Twitter, etc.
What about mobile? Google’s share in mobile is even higher thanks to Android and also the iPhone. Do the results hold up on mobile?
The same engine that delivers the results that are behind this campaign, those results also show up on mobile devices, too.
Obviously Google is not sitting on its hands. They are claiming countless search improvements per quarter. They recently launched something they call a Knowledge Graph. You’re testing this on a day-to-day basis, and on an ongoing basis to make sure that whatever gains you made are sustainable?
Volume of announcements is not necessarily correlated with sort of big movements in search quality. We welcome a debate with those guys on search quality. That’s a healthy thing for the industry, and certainly for customers. We’re confident enough in the results to actually have a tool and make it available on our site.
