Google is trying to become a more visual, more exploratory search engine

Google is trying to blow up how you think about search. To say it’s pivoting to compete in a world where TikTok and Instagram are changing the way the internet works would be an overstatement… but not a big one. Google now exists on a more visual, more interactive internet, in which users want to be surprised and delighted as often as they just want an answer to their questions. In that world, what is a search engine even for? The Google you see tomorrow might not be completely different, but the change is already starting.

At its annual Search On event today, Google showed off a bunch of new ways for people to search the internet. Most of them continue the trend of Google’s last few years: trying to find more natural and more visual ways for people to input searches and get results. You can now ask Google a question by taking a picture or rambling into your phone’s microphone rather than trying to type the perfect set of keywords into the search bar. And Google is looking for more ways to present information you might care about without you even having to ask. 

It’s an interesting thought experiment, really: what would Google’s equivalent of TikTok’s For You page look like? Google’s search team doesn’t know exactly, but it’s working on it. And at least so far, it looks like the answer will start to appear on the homepage of Google’s iOS app. That’s where many of Google’s new features are getting their start and where lots of customers are already interacting with Google in new ways.

In interviews ahead of the event, Google executives said over and over that search is undergoing a total reinvention. For two decades, “roughly the rules of the game are, ‘Dear human, if you follow the rules and script your queries just right, we’ll give you amazing answers to your needs,’” says Prabhakar Raghavan, Google’s SVP in charge of search. “But thanks to incredible — and frankly unprecedented — advances in AI and machine learning and computer vision, the tables are turning now.” 

Those advances in AI and computer vision are what power Google Lens and the new Multisearch feature with which you can search with a picture and then modify it with text. (Google always explains this with a dress — snap a photo of the green dress you like, type “in purple,” and you’re off to the races.) Multisearch has been available for a few months and is now rolling out globally. Google’s classic list of links is starting to change, too, to be replaced in some contexts by a mosaic of images and informational widgets. (Sometimes links are still the best answer, Google thinks, but not always. Not even usually.) Google’s also expanding its Immersive View in Maps, which gives you a fly-through visual view of a place before you actually go there. Google’s inputs and outputs are both becoming more multisensory over time.

Underlying the announcements, there’s a big shift happening inside of Google’s search engine. The rules of the game Raghavan describes have always relied on the idea that there is a single right answer out there somewhere — and all you had to do to get it was ask Google the exact right question. But increasingly, Google is embracing the idea that search isn’t a question-and-answer system. It’s a system for exploration, for discovery, for trying to learn things about which there are no obvious answers. And that changes both what users want from Google and the responsibility Google bears in what it decides to give them.

Liz Reid, the VP who oversees all of Google’s search products, says that people have been using Google in these more indirect ways for a while. Yes, of course, most people still come to answer a question or find a link; words like “Facebook” and “Weather” are the most popular search terms by a long shot. Others come on a longer but more directed journey — they want to buy a bike or learn about the history of the onion. “And then sometimes, people wander,” Reid says. 

The wanderers come to Google with much less direct intention. They heard a term they didn’t know; they were talking to a friend about a place that sounds interesting; they want to know more about Adele. These are the folks for whom TikTok is a surprisingly useful search engine, the young internet users that Raghavan says experience the internet in a more natively visual way. Google’s own discovery engines — the feed in the Google app, the one off to the side of your homescreen in Android, Google News, and others — are already extremely popular, and Google’s trying to bring some of that same energy to its most important product.

One way that appears in practice is a new way of thinking about autocomplete. In Reid’s demo, a user types in “Best,” and a few suggestions appear just below, with what Google calls “chips”: tappable buttons that add something to your search results. “Best” offers “Buy,” “Movies,” “Restaurants,” and a few others. Keep typing, and “Best Mexico cities” offers “to retire,” “for expats,” and more. In some cases, Reid says, the chips’ goal is to help specify your search to get the result. In others, it might broaden your search or suggest something ancillary. In both cases, Google’s AI lets it switch from a syntactic search, simply predicting the word you’re going to type based on the last one, to a semantic one, actually understanding the content and context of your search.

Once you’ve landed on the (also increasingly visual) results page, Google’s also pivoting away from a ranked list of links into (it hopes) a tool for broadening your horizons. “If you think about our ranking, conceptually,” Reid says, “as you scroll more, it gets worse.” Tons of links used to be helpful, back when Google wasn’t reliably bringing the best thing to the top. Now, Reid says, Google’s good at that bit. So as you scroll, “probably what you want is not a slightly worse version of the same thing but actually something slightly different.” Going forward, the bottom of your results page might be a batch of results for a related search about the same topic or about an adjacent subject. 

The standard Google search behavior has been the same for two decades: type in a search, scroll through the results, and if you don’t like what you see, try another query. The only tool you had was your keyword string. Raghavan says that’s exactly backward: “It’s sad when our users blame themselves,” he says. “If you didn’t get what you want, we have a problem, we should fix it.”

This all gives Google even more control over what you see in your search results. It raises long-standing questions about the bias in Google’s algorithms and its permanently mysterious ranking system but also about Google’s own business model. The company has spent recent years steadily keeping more results for itself, redirecting users to other Google products or simply putting the answer right in the results. And now, the company is beginning to make many more proactive decisions about what you see and when. It’s not just offering related searches — it’s guiding you to new topics and putting big tappable buttons right under the search box telling you exactly where to go. The Google search box used to be a blank white page — famously known as the internet’s most valuable real estate — and now it’s sending you signals from practically every pixel.

When it comes to questions of misinformation and problematic content on the internet, Raghavan is resolute that those things are mostly not Google’s to litigate. “We promise universal access to information,” he says, “which means if it’s on the internet, unless prohibited by law or some really restrictive policy, right, we will show it to you.” He talks frequently about questions with many opinions but no set answer and says Google’s job is to surface useful things but ultimately let users decide. The phrase “authoritative information” is a favorite of his and Google’s in general.

And in some cases where people disagree but there’s a clear truth to the matter? He acknowledges that it’s hard. “If you ask me, ‘Were ballot boxes stolen in Arizona?’ I cannot tell you yes or no because ballot boxes in Arizona do not sit on our indexes.” Ultimately, he seems to be saying, Google only knows what it knows, and all it can do is not pretend otherwise. This has been a challenge for Google in the past — the company has occasionally surfaced wrong information into the answer boxes at the top of the page or ranks misinformation high in results — and Raghavan says that work is always ongoing.

There’s a tricky user interface question in that, too. Google’s original “10 blue links” results were imperfect but hugely information-dense; swapping those for a full-screen image or a link to a video is simultaneously both richer and less scannable. And as Google gets bolder in showing you information before you even ask for it, it will have to also get better at explaining what you’re seeing and why, all without overloading the page. “It’s definitely a challenge,” Reid says. “How do you make it simple enough but also obvious enough?” The same goes for the search box: “Empty boxes can be incredibly empowering and simplifying,” she says, “and in other cases, they can be like, ‘What am I supposed to do with this box?’” On the search box side, the goal is to essentially let anyone do anything and trust Google to figure it out. The results bit is more complex and ultimately even more on Google’s shoulders to get right.

Many of these projects are still in the exploration and testing phase, Reid says, and she’s curious to see what many of them turn into. Google moves slowly, especially when it comes to rethinking its core business. But she’s convinced that Google can be more than just a responsive answer machine. 

She uses woodworking as an example: if Google knows you’re interested in woodworking as a hobby, how can it help facilitate that? By answering questions, yes, but also by showing you new tools you didn’t know about, cool YouTubers you’ve never heard of, places you can go to learn new skills, and much more. You won’t get all that with even the most skillfully crafted query, and 10 blue links won’t get you there, either. Google’s still the world’s preeminent search engine, but if it wants to take on TikTok and Instagram and remain the world’s portal to culture and information, it’s going to have to be much more.