If Google Is Dying, Here’s Why (Survey Results)

Google Search is under attack by its users. Readers of Reddit (well, readers of some subreddits) have said, to borrow a line from Duke Ellington, that Google search ain’t what it used to be.

The proof, writes blogger DKB (a software engineer who’s name is Dmitri Brereton), is found in the intersection between Google and Reddit. He wrote in a blog post

“Reddit is currently the most popular search engine. The only people who don’t know that are the team at Reddit, who can’t be bothered to build a decent search interface. So instead we resort to using Google, and appending the word ‘reddit’ to the end of our queries. Why are people searching Reddit specifically? The short answer is that Google search results are clearly dying.”

Response to his blog post was swift and supportive on Reddit and Hacker News, and the post was soon cited by The New Yorker (What Google Search Isn’t Showing You by Kyle Chayka) and The Atlantic (The Open Secret of Google Search by Charlie Warzel) in their own articles on Google search’s diminishing usefulness.

To sum up Mr. Brereton’s argument, Google search results are inauthentic because “the web has become too inauthentic to trust.” Ecommerce and ad-supported sites, seeking traffic to generate their own revenue, cracked Google’s secretive rules for ranking results. These sites are engineered to be shiny objects for Google’s crawler, generating traffic for themselves but providing little information of value to the actual humans who click through. 

I’ve found this to be true as well, so I wondered: If Google search is getting less-and-less useful, is there just one reason or are there others? What, in addition to inauthentic results, makes for a bad search experience? Do people who like how Google performs for them have a different view than people who don’t? 

I scoured the internet (using Google) to see what people thought. I walked away with lots of reasons, but no leading cause. To find a precise answer, I turned to pairLab by asking users: “Which aspect of Google search frustrates you more?”

How This Survey Was Conducted (Methodology)

Google users told pairLab which aspect of Google search frustrates them the most in a public survey run between April 21 and May 4, 2022. Survey participants were found on Facebook, Reddit, Twitter, and through my personal network. There was no incentive offered to participants. 

Seventeen people provided votes within acceptable parameters (thank you, participants), and voted on 66 theoretical idea matchups for a total of 412 pair votes.

What aspects of Google search did I test? I boiled the criticisms I found down to 11 specific ideas (here in alphabetical order):

  • Dated design
  • Difficult to tell the difference between real results and ads
  • Google doesn't understand what I really mean or want
  • Google gives me results for content behind a paywall I can't access
  • Google ranks its own products first
  • I must tell Google what sites to search
  • My search follows me as ads on other sites
  • Results after the first page aren't useful
  • Results are manipulated by SEO, are inauthentic
  • Results link to sites/videos that are really advertisements
  • Search results pushed down by 'People also ask' and quick answer boxes

Of course, being a pairLab expairiment, which crowdsources ideas, participants were invited to suggest their own ideas. One idea (“too many ads”) was fed back into the survey to be voted on by subsequent participants.

Segments

pairLab buckets survey results into participant segments. Segmenting participant data allows us to see what each group of people feel is important, and to compare segment results. Comparing segments shows how each segment prioritizes ideas differently from the others, leading to deep insights.

This survey asked a single segment question (the percentage of participants is shown in parentheses):

  • How well does Google search work for you?
    • It works great for me (35.3%)
    • I don't love it but works well enough (41.7%)
    • It does not work well for me (17.6%)

The deep critics are in the minority for this expairiment. Does that mean that the inauthenticity of results is not really an issue, as DKB and others suggest? Let’s find out.

Results

If Google search is dying, what are the symptoms of its illness? Inauthentic results, it turns out, tops the list, but it’s not the only cause of user frustration.

Here are the results for all survey participants:

Survey ranking chart, details are discussed in the text (from results for what frustrates you most about Google search)

There are lots of interesting things going on here. First, here’s how to read this chart:

pairLab shows rankings in two dimensions. The first is an ordinal ranking (Google users ranked “Results are manipulated by SEO are inauthentic,” as the idea that most frustrates them about Google search). 

The second dimension reflects emotion and is expressed as a “pair score” (the number assigned to each idea). A pair score is a unitless measure that shows how strongly survey participants prefer one idea as compared to any other. 

We can use pair scores to figure out how passionate people are about each idea. For example, we can say that the top idea (“results are manipulated by SEO”) is about twice as frustrating as the second-place ides (“Results link to sites/videos that are really advertisements”; 21.8/11.1 = 1.96), and nearly eight times more frustrating as the last-place idea (“I must tell Google what sites to search”; 21.8/2.8 = 7.8).

(By the way, green bars show rankings above the mean for all ideas, gray bars show rankings below the mean.)

Looking at these data, Google users as a whole don’t share Mr. Brereton’s frustration of having to tell Google what site to search, but they resoundingly agree with him that Google is getting tricked by sites that have decoded its secrets. (To reiterate the math: all users surveyed said inauthentic results were about eight times more frustrating than having to tell Google what to search. Using the graph above you can explore other differences by simply dividing two pair scores.)

How about the different segments? Do they all agree, or do each have their own concerns?

Let’s compare them to find out.

When we compare at the results from people for whom Google “works great for me” with the “it does not work well for me” segment, we see that that the two agree where Google search frustrates them more than average:

  • Results are manipulated by SEO, are inauthentic
  • Google ranks its own products first

They also agree that these aspects bug them less than average:

  • Dated design
  • Results after the first page aren’t useful
  • I must tell Google what sites to search

To be sure, the two user segments agree on only a third of the ideas. They disagreed on the remaining eight aspects, so there is a gap of perception between the two.

How about comparing the works great to “works well enough?” We see a lot more agreement, in fact they agree on 75% of the ideas. Here’s what bugs both segments most:

  • Google gives me results for content behind a paywall I can't access
  • Google ranks its own products first
  • Results are manipulated by SEO
  • Results link to sites/videos that are really advertisements
  • Search results pushed down by 'People also ask' and quick answer boxes

And here’s what they agree annoys them less than average:

  • Dated design
  • I must tell Google what sites to search
  • Results after the first page aren't useful
  • Too many ads

To be complete, the only issues where “does not work well” and “works well enough” agree that  “Results are manipulated by SEO” frustrates them both more than any other aspect of Google search.

Are Google Search Results “Clearly Dying?”

This pairLab expairiment confirms DKB’s observation that Google search is frustrating because its results are inauthentic. Whether users love, hate or are accepting of Google search’s performance, all were passionate about the inauthenticity of its results, citing these ideas as the top aspects of Google search that frustrate them most. Whether results linked to sites that provided only content meant to game Google’s algorithm or were themselves veiled advertisements, users say Google is serving up ads rather than serving its users. 

Looking at this deeper, the ideas tested (which are all based on real user complaints) fall into three categories:

  • Authenticity (difficult to tell the difference between real results and ads; results are manipulated by SEO, are inauthentic; results link to sites/videos that are really advertisements; Google gives me results for content behind a paywall I can't access; Google ranks its own products first; my search follows me as ads on other sites
  • Function (results after the first page aren't useful; Google doesn't understand what I really mean or want; I must tell Google what sites to search)
  • User interface (dated design, search results pushed down by 'People also ask' and quick answer boxes; too many ads)

Authenticity problems bubble to the top for all segments. There are some variations (the “doesn’t work well for me” finds ads more frustrating than the other segments do, for example), but overall and for each, the quality of results frustrates all users more than other problems they perceive when using Google search.

Picking apart a couple of these ideas from a product view, you might argue that sites behind a paywall are valuable to users. I agree that they may have content users find valuable enough to pay for. But when Google positions largely inaccessible content on equal par with accessible content, users feel tricked and may wonder if business shenanigans between the two companies conspire to shape the results. Are these results really ads in sheep's clothing?

How about Google putting its products at the top of results? Google owns its platform, it has (in my mind) every right to promote its own products there. But aren’t these really house ads? If they are, why don’t they look like house ads? And what happens to any product Google deems a competitor?

Try this: Search for “injury lawyer” on Google. You will see “Ad” prominently displayed next to the paid search results. Now search for “analytics.” Assuming we’re looking for digital analytics, there are more than a dozen leading vendors that compete against Google Analytics. Would you know that from the page you see? No mention of Google Analytics is tagged as an ad, but they appear to act as exclusive ads.

So, also from a product standpoint, Google search has an authenticity problem. Lack of authenticity erodes trust, and that opens the door for a disruptor. 

If I were building a search engine that in some way competed with Google, I’d target the authenticity of their results. 

Consider DuckDuckGo. This Google search competitor’s value proposition is “privacy, simplified.” From this pairLab study, privacy was never raised as a frustration issue (I did not find it as a broad complaint nor was it suggested by survey participants). If DuckDuckGo were to wonder about siphoning off more traffic from Google, I’d advise working towards results users can trust. There’s an opportunity there. Someone will figure it out.

This pairLab study shows that Google search has a problem: its users lack confidence in the quality of its search results. Blogger DKB was on the right track, and pairLab shows where else it led. 

Let pairLab Help You Find a Path to Your Product Vision

pairLab is an expert tool for finding your customers’ wants, needs, pains and gains, and it helps you make informed decisions about your products and find problems that a competitor could use against you. The results are stunning. Learn more and get the free trial.

Posted by Christian
on July 14, 2022