Google Search gets new ‘search your screen’ AI feature on Android

Google recently announced a surprising press conference for February 8th, prompting speculation that it would unveil its response to ChatGPT at the show. The chatbot reportedly terrified Google because the company had yet to unveil its own ChatGPT-like products. The Wednesday event did focus on AI, but it turned out to be about core Google apps like Maps, Search, and Translate.

The apps did not receive chatbot support, but Google highlighted how AI improves these experiences. Google Search got its own AI upgrades, as the app will offer enhanced multisearch experiences. Even more interesting is a “search your screen” feature coming soon to Android phones.

Google already includes Lens support in the Google Search experience on mobile devices. The feature lets you point the camera at objects or landmarks and learn more details about them.

In the coming months, Google will bring a new “Search your screen” feature to Android. This lets people use Search to look for information connected to the data on their screens. It will work with photos, videos, websites, and apps.

You won’t have to leave the app to perform the search. Just long-press the home button on Android to bring up the Google Assistant along with the “search screen” option. It’s unclear whether a similar feature will be available on iPhone since iOS devices can’t invoke the Assistant as easily.

The feature will also work for shopping, letting you find similar products to the ones you spot around.

That’s where multisearch comes in.

You might take a photo of a product you want to buy and then type in a different color for it. That’s one way to use multisearch on phones, and it’s a feature that involves AI working in the background.

Google says it’s improving multisearch to let you search locally for products “near me.” The feature will be available in English in the US initially. It’ll then expand to other markets.

Furthermore, multisearch will soon support any image you see on the search results page on mobile. Say you find an interesting product accidentally via Search. You’ll be able to use multisearch to find something similar that might be more suitable for you (see the example above).