Is Google Chrome incognito browsing really private?

A recent study by private search engine DuckDuckGo says, among other findings, Google Chrome’s incognito mode isn’t as private or anonymous as you might expect when running Google Searches. The study, published in December 2018, states Google Search tailors search results to individual users regardless of whether they use Chrome’s private browsing feature:

“Private browsing mode and being logged out of Google offered very little filter bubble protection. These tactics simply do not provide the anonymity most people expect. In fact, it’s simply not possible to use Google search and avoid its filter bubble.”

In simplest terms, the study concludes that different people who search for identical words and phrases can get different search results, and/or the same results ranked in a different order, based on their personal data. One might expect that using incognito mode would remove any bias derived from personal data, but that does not seem to be the case.

The study goes on to say, “These discrepancies could not be explained by changes in location, time, by being logged in to Google, or by Google testing algorithm changes to a small subset of users.”

DuckDuckGo’s study focuses on what it calls “filter bubbles,” wherein users are steered toward content that already aligns with their own ideals and beliefs and competing viewpoints are weeded out or devalued. Google suppresses search results that it believes you’re not likely to click on in favor of those that you are based on personal data it has collected. The “editorialized results […] can have a significant effect on political outcomes in aggregate,” the study says.

DuckDuckGo chose politically sensitive search phrases such as “gun control”, “immigration”, and “vaccinations” for its study.

How Google Chrome incognito (a.k.a. private browsing) mode works

Google Chrome’s support documentation states,

When you browse privately, other people who use the device won’t see your activity. Chrome doesn’t save your browsing history or information entered in forms. Cookies and site data are remembered while you’re browsing, but deleted when you exit Incognito mode.

Note that incognito mode is intended to make your web browsing private from other users on the same device, not private from the websites and services you access online. However, it’s commonly believed that searching while incognito removes the identifiers that Google would use to personalize search results.

To customize results, Google needs to identify the user. Websites typically have two ways of identifying an individual user: cookies and an IP address. Chrome also lets users log into the browser to sync bookmarks, extensions, and settings across devices, a feature that could also be used to monitor individual users.

Logging out of Chrome and using incognito mode presumably removes many of the identifiers that would be stored in your browser. However, DuckDuckGo states Google Search results are still customized regardless of whether you log out and use incognito mode.

So how does Google identify incognito users? We can be fairly certain that an incognito window does not retain persistent cookies from a standard Chrome window. Similarly, logging out of the browser ought to prevent Chrome from recording activity and storing it in your Google account profile. That leaves us with IP addresses as the most likely culprit for how Google identifies an incognito user.

To test this theory, we ran a couple experiments of our own.

Does Google personalize search results based on IP address?

In short: Google localizes search results based on your IP address even if you’re logged out of Chrome and incognito, but we found no evidence that IP addresses are used to personalize results or track users at an individual level. In other words, Google uses your IP address to identify your location, not your specific device. Other perceived bias in search results could be a result of randomization, not personalization.

Our hypothesis was: when logged out of Chrome and using an incognito window, Google Search will use an IP address as a means to identify a device and customize search results accordingly.

Therefore, all other factors constant, changing our IP address will produce different results for the same search terms.

Google randomizes search results no matter what

But before we even started this experiment, we hit a snag: Google will produce different search results and rankings even if all of factors remain constant. When we log out of Chrome, connect to a VPN, open an incognito window, and search “gun control”, we get slightly different results than when we run the exact same search under the exact same conditions a few seconds later. One or two of the search results is usually different. In one example, we saw an article from the right-wing National Review, and in the next, a link to a left-leaning Aljazeera article instead.

Without reliable constants, it’s unfeasible for us to run a good experiment that would verify or reject DuckDuckGo’s findings.

It would appear that Google tests different search results and rankings somewhat at random, possibly to learn what users are clicking on and uprank successful links accordingly. Whatever the case, it makes testing for customization based on an IP address very difficult. If the results are somewhat random regardless of whether the IP address changes or not, how would we know if changing the IP address actually has an effect and that effect is a result of personalization based on user data? We’re not sure if or how DuckDuckGo overcame this obstacle in its study.

Google localizes search results based on IP address

The only clear sign of customization is based on location. IP addresses correspond to approximate locations in countries and cities. Because I’m in Canada, even though I’m searching on Google.com (not Google.ca), I still get a lot of results from Canadian websites due to my Canadian IP address.

We tested this hypothesis using a VPN, or Virtual Private Network. Among other benefits, a VPN will mask our device’s IP address with that of the VPN server, so Google will see a different IP address once we’ve connected to the VPN.

We’ve elected to use CyberGhost, which has plenty of servers and earned a perfect score in our VPN privacy and security assessment.

We recorded the domains on the first page of Google Search results for “gun control”, “immigration”, and “vaccines” under four different scenarios:

1. Logged into Chrome, standard Chrome window, no VPN
2. Logged out of Chrome, incognito, no VPN
3. Logged out of Chrome, incognito, VPN connected (Vancouver)
4. Logged out of Chrome, incognito, VPN connected (Vancouver, different server)
5. Logged out of Chrome, incognito, VPN connected (Seattle)

We found that Google produced a unique set of search results for every test. But as we mentioned before, this is more likely due to randomization rather than personalization based on personal data. The first few results are almost always the same, and discrepancies start to appear in the bottom four or five links of the first page of results.

The only significant difference came when we ran the searches using a VPN connected to a different country, in which case we saw different localized results.

You can view the results of our tests below:

It is also worth mentioning an earlier study we conducted, which found no bias against US President Donald Trump on Google’s part from a broader view of search results. That study analyzed the top search results for “trump news” and their average referring domain counts.

Will a VPN prevent biased search results?

Not exactly, but it can change what results you see in Google.

If you connect to a VPN server in another location, particularly another country, you will get localized search results if applicable. Those search results are still personalized, in a way, based on where you’re connecting from, but they don’t seem to be based on your individual IP address.

Localization aside, DuckDuckGo claims that Google personalizes search results even if you’re in incognito mode. We didn’t find substantial evidence of this, but even if true, that personalization wouldn’t seem to be based on individual IP addresses.

If you want truly unbiased search results free from personalization and localization, you’ll have to use something other than Google Search. Check out our list of Google Search alternatives for more info.

Experiment notes

I am using Google.com (not Google.ca) from Canada.

Unless otherwise stated, I used VPN servers in nearby Vancouver to minimize any variation in search results due to location. In fact, the two VPN servers I used were from the same provider in the same city).

Date of search is December 28, 2018.

Sponsored results, Twitter results, and news results are not included; the latter two are not included because they change too frequently to conduct reliable tests.

If two results from the same domain are listed, we only included it once in our results.

I opened a new incognito window for each search.

How Google tracks you online

When you use Google services and products such as Search and Chrome, the company records your activity and saves it to a profile. These profiles are associated with identifiers stored in your web browser, like cookies, or identifiers specific to your device, like an IP address. Even when you’re not explicitly using a Google product or service, so many websites utilize Google Analytics and other Google tools and plugins that Google can track your activity across the web with an alarming degree of accuracy.

When you use Google Search, Google sees the identifier and tailors search results based on the associated profile information it has collected. This tactic is quite common on the internet and is frequently used for advertising purposes. Facebook and Amazon employ similar tracking technologies, for example.

Incognito and similar browsing modes from other web browsers remove cookies and other identifiers that are stored in your browser. IP addresses are not stored in your browser and are therefore still visible when using incognito mode.

A unique IP address is assigned to every device that connects to the internet. Devices connected to wifi routers often share the same public IP address, which is visible to all other computers connected to the internet. For most internet users, public IP addresses are just temporary and change periodically, but not so often that they can’t be used to track someone. Connecting to a VPN will mask your device’s IP address with that of the VPN server.