by At Google I/O, Google Senior Vice President Prabhakar Raghavan announced new enhancements to its Lens Multisearch tool, which lets you perform a search using just an image and a couple of words. A new mode, called “near me,” will allow users to take a photo of an object and then find the results locally.
As Raghavan explained, you’ll be able to take a photo of a dish and then search for restaurants that serve that specific food. Google will then display a list of relevant restaurants near you. For this feature to happen, Google scans relevant photos from locations near you and then compares them to the one you uploaded. “Near Me” will be available in English to start with and will expand to more languages ”over time.”
Google is also implementing something called scene scanning. This will allow users to pan their camera and then enter a search phrase about objects in front of them. In explaining the feature, Raghavan used the example of trying to find a nut-free chocolate bar in a supermarket. He’ll be able to scan an entire shelf of candy bars and then see overlays that provide “helpful insights” like reviews on each item. We think Raghavan’s description of the feature sums it up pretty well: “This is like having supercharged ctrl-F for the world around you.”
The search giant first launched multisearch in April, but its purpose was mainly related to shopping and finding instructions. For example, you can take a screenshot or picture of a dress you like, and then type the name of a color. Google multiple search will show a list of results with similar dresses in that color. Or you can take a photo of a specific type of plant and search for “care instructions” to find relevant information.
But, as Lou Wang, Google’s director of product management, hinted during its launch, Google’s multi-search feature can be used for so much more, and it looks like that extra functionality is starting to arrive.
To use multi search, open the Google app on Android or iOS, and then tap the Lens icon on the right side of the search bar. From there, you can upload a photo or screenshot from your gallery, or take a photo of an object in front of you, to begin your search. Then swipe up on the screen and select “Add to your search” to search for the photo along with a relevant word or phrase. It’s not yet entirely clear how Google will integrate near me and scene browsing into its interface.
Update 1:51 p.m. ET: Updated to add additional context from Prabhakar Raghavan.
