I had a front row seat for this test. I believe the experiment we ran provides conclusive proof. I'm on a panel with a representative from Bing later today and I'll ask Bing about this directly.
It's embarrassing for Google to complain about this. You FINALLY get a little competition on your turf and you try to make some big issue that, as a market leader, the product you produce is being watched, analyzed and in some ways incorporated by your competitor.
There is no victim here. They are not taking your 1st result and copying it. They are taking the result the user clicked. Obviously you didn't predict that with your algorithm or you'd have always made that the 1st result. Instead, what they're tracking is user behavior, not your raw ranking.
Obviously users give Google implicit permission to track their behavior by using your product. And similarly, by installing the Bing toolbar, they're giving Bing that permission.
In short, Bing Toolbar infers relationship between words on the page and the next page the user clicks on. Google's team purposefully confined Bing Toolbar behavior-tracking algorithm to their use of google.com search result page, and then cried fool about "bing stealing Google's search results".
This is disgraceful attention-whoring on Google's part. Quite surprising, too, as I don't remember them ever stooping that low.
They may just be using terms in the referring URL rather than the referring page. It'd probably yield better data. Most site-specific search pages would have the search term in the URL.
I wouldn't be surprised if they're more interested in data from domain-specific sites like epicurious than generic search sites like Google.
I'd also guess that this won't work once the SEO guys figure out they can feed fake clickstreams to MS.
Referring URL? Oh right, Google intercepts all search result clicks through a redirector, and includes the search term in the intercepting URL. Yeah, the unique words in that URL could figure into describing the content of the final destination page (after redirect), the same way anchor text figures as well.
Even Matt's allegation is softened by "I believe" here: there appears to be nothing that conclusively indicates Bing is solely targeting Google. For example, the observed behaviour could be a side-effect of a generic algorithm to extract and associate search queries with a user's click stream, which is only a minor variant of what Google itself does with its own toolbar.
If the case described above were true, then all Google has done here is to make inconclusive accusations and use the occasion to highlight its own dominance over search.
It seems to me this is just a cheap and slightly seedy PR stunt.
Associating search queries and click stream behaviour is fine if it's your own search engine. Doing it for someone else's search engine isn't (regardless of who's doing it).
In the hypothetical situation above, it has almost nothing to do with the search engine - it is both the user providing the query and selecting the result: this is the data of value, not which intermediary provided the list of results to select from.
Really? Doesn't the intermediary that narrowed the list from several billion possible matches to the best 10, including correcting the inherent spelling errors in the query carry some value?
> Associating search queries and click stream behaviour is fine if it's your own search engine. Doing it for someone else's search engine isn't (regardless of who's doing it).
They have a very long history of playing dirty. Lotus learned it, MS took information Lotus shared with them and then shared it with Excel and Office and they supposedly kept Lotus on an API changing treadmill. Digital Research learned it, MS wrote code that made Windows 3.x crash if it detected DR-DOS. Netscape learned it. Arguably, IBM(OS/2) and any other operating system vendor learned it in the 1990s as well, MS charged premiums if hardware vendors wanted to install non-Windows operating systems. They sort of tried to do it to Intuit, they made a competitor and then effectively gave it away for free. Enough so that a lot of folks avoid Mono like it's, well, actually mono. They've established that reputation, and most of the time, by the time it became clear what was going on, MS had already done irreparable damage.
It is kind of embarrassing for Google, but if it is real and it continues, it's better to address it now rather than after MS becomes a titan of search and Google's market has eroded. At times, it seems like MS has changed in ways, but fundamentally they're still run by the same guys. Remember that when you play your Xbox or use Bing or any MS products, they don't like to see other successful software companies.
Also there's the strange fact that Bing and their whole online division makes gigantic losses. They're not in it for the money, they're in it to stifle competition and hold back progress so they can milk their cashcow some more.
I think they are complaining because it could be far more widespread: it would actually be easier for the head than the long tail. Where it‘s harder is for News, and Bing appears to lag for recent results.
I remember that when Bing went out, everyone was wondering how close to Google the results were (and talked about it as a good thing).
If it's beneath Google to complain about Microsoft riding on its coattails for the highly valuable "long tail" of queries, surely it's beneath Microsoft to sue Android manufacturers for competing in the smartphone space?
Would it have been better if Google had jumped straight to the questionable lawsuit part, like every other company seems to do when threatened on its own turf?
Let me try to understand what happened. For some obscure searches that return no results normally, a handful of users searched on Google using IE's toolbar and then clicked on bogus results.
Those bogus results made it to Bing's results eventually.
Ok... So it proves Microsoft analyzes the toolbar behavior and when it has no other data, it will therefore look like a copy of Google search.
Sounds fair to me. Do you want to get into a discussion on how exactly Google tracks you online?
in 7-9% of the times google tried to spam bing. not in 7-9% of search results. this indicates that maybe google isn't that great at figuring out how to spam bing or that bing is pretty good at defending agains spam. maybe google could take some lessons from bing on cleaning up spam and problem that seems all to prevelant on google these days.
There is, however, the legitimate complaint that they apparently do not finally have a little competition on their turf. Their competition is cheating, not innovating. That helps who, how? At best for "competition" sakes, Bing nabs a big share of the market; now there are two big dogs who make it hard to enter into the search realm with new ideas.
As we all know, this isn't the first time Microsoft has copied someone else. And I'm sure it won't be the last.
I think Google has a right to complain. Microsoft has resorted to these less than innovative tactics to monopolize themselves for a long time now, and it isn't fair to companies like Google who have worked their butts off (and gave 1.8 million shares - $336M in 2005 - to Stanford for the PageRank algorithm) to develop their superior product.
You clearly didn't read the whole article. If you did, you clearly didn't understand the article. Step back a little bit, fanboy. Microsoft wasn't copying anyone here.
Oh I definitely did read it in its entirety and understood it perfectly. What you're failing to do is see the whole picture.
Let's put it this way... if Google hadn't bought the PageRank algorithm from Stanford and put years of work into perfecting their search results, Microsoft wouldn't have any way to track which Google search results users click. It's an unfair tactic that clearly demonstrates Microsoft's sketchiness and desire to monopolize themselves (by any means necessary, "evil" or not) wherever there's a computer.
As for the fanboy comment... I'm certainly not a fanboy but I'll let the following speak for itself: Microsoft Internet Explorer vs Google Chrome
All your comments are coming from your assumption that Microsoft is trying to monopolize in something - in this case, search. Hence, your comments (although you will disagree) are biased and irrational. Microsoft isn't trying to monopolize in anything nowadays. In fact, they can't, so they aren't even trying.
From a search engine user's point of view, I believe this whole fiasco is ridiculous. First, it's ridiculous because Google is handling this situation very immaturely. Matt Cutts should not have confronted the VP of Bing in a way he did. Second, if I were the user of the Bing Toolbar, I gave permission to the Bing Toolbar to use my behaviors to polish my search results. I have no problem with that. Lastly, the experiments they did has more to do with "guessing what user wanted" than "what PageRank does".
I've used Bing fairly often past 6 months because of too many spams Google search results were giving back. Now that Google has fixed (or still working on) the spam problem, I'm starting to use Google again. However, what I noticed from the past 6 months is that Google search isn't so much better than Bing. This Bing Toolbar fiasco only applies to synthetic queries that I would never make.
Is Bing cheating? I don't think so. To me, they are just using another signal from user's permission. However, the definition of cheating will be different for everyone else.
Err, by that line of thinking, Google leveraged Linux (the hard work of volunteers) to earn tens of billions and does not release the modified code for use of the volunteers. Of course they are not required to, but it isn't fair to Linux developers who have worked their butts off to develop Linux.
>It's more like Linus say you can't use the code, but Google use them anyway.
By installing the Bing Toolbar, users are giving permission to track their clicks. If Bing's server farm is searching Google and parsing the results then it is more like your example.
They employ Andrew Morton and Ted T'so, and have been working pretty hard to get the delta between their custom Linux and upstream down- I'd say they've been pretty fair to Linux developers.
I have a suspicion what you'll find is that Bing use the toolbar to match $current_page_content with $clicked_page_content. When $current_page_content contains obscure words, that becomes the only signal, and so bing's engine will naturally associate it with $clicked_page.
In other words, there's a relationship between Page A and Page B if there exists a link beween them (==PageRank). But the strength of the relationship is increased based on how many users click on that link. I think that's the information Bing were trying to capture (or if they weren't, they should have been).
What I'm saying is that it's probably an unintentional side-effect. At scale though, the effect is that Bing gradually uses Google as a signal, simply because Google is a popular site.
edit: Yet another way of saying it: I think it's not just clicks on Google searches that are captured by Bing, but clicks anywhere. Google is a large site, so its influence on Bing can be measured. This is what we're seeing. My theory. I don't work in search.
Exactly. If they are just matching (even more simply) $search_term_entered to $clicked_link then you would expect that they are "copying" from any search engine configured in the toolbar.
Now the interesting thing to reverse engineer is what other information might be passed along to give relevance to the search term/click pair. If Google could establish that there was a third piece of info in the tuple, such as "originating search domain" and that Bing used this to weight term/click pairs based on the authority of the source, Google's claims would hold more water. I suspect that Bing has to apply some kind of validation of the term/click pairs (for instance, only sending pairs that appear on the same results page from accredited engines), otherwise they would be subject to "Bing bomb" attacks where users or botnets vote up lower ranked (or even unranked) clicks for a given term. (And if they don't validate or detect gaming, then there would be ample opportunity to inject all kinds of synthetic behavior into Bing's search results. Based on the relatively few number of users and clicks it took to own a long tail term, it seems like the protection they have is very weak or simple.)
This makes a lot of sense, and would have be easy enough for Google to test as well, creating some tiny, brand new, never before heard of test search engine that Bing would have no reason to copy, see if the same thing happened.
edit: I'm not even sure if it's only search engines that are being analysed by Bing or all pages, but it's possible that it is just SEs - they could be capturing query terms distinctly.
Also, why did the experiment succeed for only 6 or 7 of the 100 terms that they tried? There's more than what meets the eye here, regardless of the hype and everyone jumping on the bandwagon.
The article suggests that the Bing toolbar monitors what its user click and uses that information to improve Bing search results. Is that what you have conclusively proved?
I'm interested in another experiment. If you set up a honeypot, search for the term, but never click on the link, does the honeypot start showing up in Bing? The article doesn't say whether you tried this. Did you try it? Are Bing scraping your results from the page or only tracking their users clicks?
Anyone can test that Microsoft's software sends the clicks back to Microsoft, although I believe Microsoft sends the data back by SSL, so it's harder to verify even that than you'd expect.
Google's search results are blocked in robots.txt, so I don't believe Bing has been able to crawl our search results directly. All the evidence points to users' clicks on Google, which are then sent to Microsoft.
Microsoft has (so far) declined to admit whether our allegation is true. Getting them to talk about exactly what they do and what software they use or don't use would be the easiest way. I'd like them to confirm or deny, which is why I wanted to go to this search panel later today and ask them.
> so I don't believe Bing has been able to crawl our search results directly
Isn't compliance with robots.txt more of a voluntary thing?
I'm not accusing MS of ignoring it when convenient, but if you/we/someone is accusing them of acting unethically wrt search results in the first place, telling the crawler to ignore robots.txt wouldn't be that far away, would it? (And likewise faking the user-agent, etc.)
For better or for worse, UA identification, robots.txt compliance - all those things are voluntary. I'm not suggesting they shouldn't be, but it certainly makes a difference in terms of whether something's possible or not. (And, if you ask me, places an even higher obligation on the actors to behave ethically, lest trust completely evaporates and the whole thing goes to hell in a handbasket).
I am not a lawyer, but as I understand it there is some precedent in the US of intentionally ignoring robots.txt being unauthorized computer access, exposing you to all the liability that entails (possibly criminal).
Internet Archive was sued unsuccessfully. As I understand it a lawsuit is still in process against Google on the topic. So I guess the precedent is weaker than I thought, but still: tread carefully.
If you read the article and other comments here it's been made perfectly clear that the Google toolbar and Chrome browser are not sending similar data back to Google.
Ah, at least the google toolbar does. If you enable PageRank on the Google Toolbar it sends back all the urls you visit just like the bing toolbar.
From the toolbar privacy policy: "Toolbar's enhanced features, such as PageRank and Sidewiki, operate by sending Google the addresses and other information about sites at the time you visit them."
Google has managed to demonstrate one way MS appears to be using the data. What does google do with their trove of data? That's a lot of data to collect and not do anything with.
If they want to make it perfectly clear they should add into their privacy policies and EULAs.
Yes absolutely. I don't think anyone in this thread or in the article denied that the Google Toolbar sends data to Google. And you are absolutely right that Google's use of the data collected should be clearly stated in a privacy policy and EULA. It might be, I haven't read them.
I did not say "similar data" because "similar" is a bit too slippery a word in a technical context. There's too much plausible deniablity. What I am asking is if Google's tools send data back to Googleplex to be mined for the sake of search engine improvements.
Quote from the article: "In fact, Google stressed that the only information that flows back at all from Chrome is what people are searching for from within the browser, if they are using Google as their search engine."
I'm pretty positive that's not true. If you run Fiddler when browsing with Chrome you will see constant hits to toolbarqueries.clients.google.com whether you're using Google or not. I could be browsing some MS site and toolbarqueries.clients.google.com gets hit. Chromium doesn't do this.
Edit: You can uncheck everything under privacy and it will still send those requests.
Edit2: What it sends back looks something like this:
Looks like auto-fill data, but this happens when I click around a site, NOT when searching Google or typing something in the address bar. For some sites (interestingly, not all) it sends 3 requests for each page load.
That's troubling. I'd be very interested in seeing a response from Google about this. Are you aware of any? Also, can you use Fiddler to inspect the content of the requests? I'm not familiar with the tool.
I see this too, if I have autofill enabled, and at least one autofill address entry.
I would guess that Chrome is sending a hash of the <form> (perhaps URL + method?), plus a hash of each of the <input> tags, and Google returns some sort of information about what kind of form it is?
If so, it would mean it's pretty easy for Google to determine which sites you're on from the pattern of hashes sent for each site. e.g. I see this data sent in the clear for pretty much every page on https://www.facebook.com/
>I believe Microsoft sends the data back by SSL, so it's harder to verify even that than you'd expect.
Please. Adding my own SSL cert to my own laptop is not harder than I'd expect. Certainly not harder than many other things you did in setting up this experiment.
poacher69, we crawl the public web. Anyone that blocks us out with robots.txt, we won't crawl. If you check bing.com/robots.txt, it has "Disallow: /search" . So no, we won't crawl Bing's search results pages. If anything, users tend to complain when search results from Lycos or wherever show up in Google.
I was gonna call out Matt for crawling bing's search results but I'm guessing Microsoft hasn't realized they return results from the /Search/ folder. ;)
From my experience, Googlebot doesn't crawl pages that are blocked in robots.txt files. Check out Bing's robots.txt: http://bing.com/robots.txt - notice how /search is disallowed. That typically means that Googlebot isn't able to access that page. The same for the other search engines, it's more down to if they specify (through robots.txt) that Googlebot isn't allowed to crawl those results.
Given that Google appears to have an active program to monitor the results of search queries on Bing and to track Bing's page rankings and the ways in which they change over time in (How else is this more likely to have come to their attention?), they should hardly be shocked, shocked to find Microsoft doing something similar.
[edit] I have always suspected that the real value of Bing for Microsoft is to prevent Google's data mining of queries originating in Redmond.
... and feeding that data back into their own search results, rather than just using it for analysis to see how your competitors are performing?
If I'm in the business of giving horse racing tips and I read your tips to see what your strike rate is compared to mine, that's one thing. If I start tipping the same horses as you, purely because you tipped them, that's quite another thing.
Microsoft isn't using strike rate data in the important sense which you imply - the strike rate for Google is advertising revenue. It's not as if Microsoft is collecting info on the advertisements displayed and then soliciting those advertisers to spend their dollars on Bing (at least that's not part of the allegations). I am pretty confident that Google feeds every bit of legally collected relevant data back into their search algorithms.
no, the "strike rate" is precision/recall. Good advertising CTRs is a side-effect of relevancy. Relevancy is measured by precision and recall.
edit to expand: If the measure by which a search engine evaluated itself was advertising revenues, they'd all have massive intrusive adverts, and no users. The only viable measure can be the quality of the search results themselves. As a happy coincidence, if you build something capable of delivering high quality results, you can very easily use that to produce highly relevant adverts. Imagine that each advert is like a little webpage, and rank them just the same as you do for normal webpages. (caveat: there's no link graph for adverts, so we're reduced to using a simpler text mining approach, eg bag of words vector space la-di-da).
I believe that Google and Microsoft evaluate their search engines by entirely different measures. Google primarily by advertising revenue, Microsoft primarily measures by preventing searches using Google. Sure the ad money is nice for Microsoft, but they don't need it for their business to be profitable and they would be doing major research into search anyway because of its importance to businesses - they sell databases after all.
This whole episode points to the sort of counter-espionage operations the two companies are engaged in. Look how important a propaganda victory is for Google? It strains credulity to believe that the release of this information on the day of the panel discussion is pure coincidence.
Microsoft, historically, have been kings of the desktop. With the rise of the web (which post-dated MS's rise), the desktop has become less and less relevant. Google is fast replacing them - my email, documents, search, advertising, analytics is all handled by Google. I don't use Microsoft for anything in my day-to-day life. Even on my main windows machine, my files are in my dropbox, outside of MS's control.
MS are desperate to regain control. Google will soon launch their own web-centric OS properly, and bam, MS will have no business apart from selling to an ever-dwindling number of companies who can't believe MS don't rule the roost any more. In 20 years they will simply cease to exist if they can't come up with a world-beating online product and win back control of people's computing lives.
Notice how they're diversifying into games and search in order to prepare for the worst case; that their core OS and 'boxed software' business fails.
I think you may misunderstand my position. Bing is basically a research project for Microsoft. The ad revenues don't really matter, the data they collect does - it's learning v. earning. It makes sense for Microsoft to spend a billion dollars because better search algorithms have application for their B2B products and services. If they recover some of their R&D costs directly through advertising revenue, that's a windfall to the overall bottom line.
Bing copying Google's search results is just evil. Not like copying Apple's iPhone design and user interface and giving it away to Apple's competitors, which is good. Right?
It's ugly and immoral and probably legal. Good job catching Microsoft at it (and I think the really really unethical and scary bit is that Microsoft is cheerfully stealing info from users via their browser). I also realize that Google got where it is in Search by innovation and iteration, and that Google's search team has nothing to do with Android per se, but you might see how Apple people feel about the business empires built on stealing their ideas.
I don't think Android devices being similar in some ways to the iPhone is even remotely analogous to what Bing is doing here. One is called healthy competition (and I don't see how Android is copying iOS). The other is literally just copying data.
They are similar in that they both are big touchscreens with no physical keyboard. Apple did not invent that by any means. That is also a shot of the Android App Drawer and not an actual homescreen. Most Android homescreens I've seen have a few widgets on them and do not look anything like a big grid of app icons like the iPhone. Also, that "original" Android phone design you point out looks a hell of a lot like a Blackberry. Android must have blatantly copied RIM by your logic.
I usually criticize Microsoft, but I am on their side in this case.
Why shouldn't Microsoft be using this kind of data? Google search result pages are part of the internet just like any other publicly available web site. Microsoft monitors what the users are clicking on Google and probably on Bing and other sites. So what? Monitoring users is not a new thing. It may be unethical and I may personally hate it, but almost everyone is doing it.
Google should stop whining about this and make their search result the best they can. If they had the best search engine, Bing could come close, but never overcome Google by just copying part of it.
Great panel session. One of the best I've seen. I think you made your point, although I think MS did a good job neutralizing it too.
My question for you Matt... is there any way for Google to build a toolbar that effectively does what the Bing toolbar does (or even a joint one?). I jump to use the various search engines because no single search engine is sufficient. But clearly when Google isn't sufficient, you don't get the value of when I go to Bing. And vice-versa (as I don't use the Bing toolbar currently, but that may change now). Or do you feel that with 65% of the market, you don't need this info?
I can't argue about the legal vs illegal. I sure would like to tell the bing team that they should better google in their attempts. This strategy is mere copying into- 'look, I am as good as google'. Microsoft never learns from past do they??
I can't help but feel a little surprised that Google found it ethical to lie on their search results page for any reason; and clearly this trojan horse page was a lie. Certainly at every search engine I've worked at we always said, "We can put up adds and help, but we can never outright lie." I suppose that's ameliorated by the fact that this was an internal experiment, but still..
