Visual search: what is it, and how can SMEs get on board?

Visual search: what is it, and how can SMEs get on board?

Charlotte Jenkins

Charlotte Jenkins
30th July 2019

Visual identification and how the brain recognises images have been a central focus of traditional marketing for decades.

The brain identifies images 600,000 times faster than it does text, meaning that we can identify an object within 13 milliseconds of seeing its image. This has a huge impact on how businesses place their products to maximise sales.

With the recent advances in machine learning, visual search, like voice-based search, is becoming the next big thing for online retail.

Unlike traditional search which relies on queries being typed into a search engine in exchange for a list of URLs, visual search allows consumers to use an image as the basis of their search. That image is then matched against other similar items and the search results show a list of visually similar products.

Whilst it’s still early days for visual search, there are a number of compelling reasons why brands are starting to consider bringing it into their marketing and sales strategies:

  • After three days, customers retain 65% of visual information compared to just 10% of auditory information.
  • Consumers are 80% more willing to engage with content that includes relevant images.
  • Content with relevant images earns 94% more views than without.
  • Images are the biggest purchase deciding factor for 93% of consumers.

(Source: Search Engine Journal)

Visual search explained

Visual search has been happening for a long time. In fact, according to research by Moz and Jumpshot, 27% of all searches are for images.

In traditional visual search, a search query is typed in and the results show up with an option of viewing the images related to that search enquiry. Here, the search engine isn’t understanding the image itself, just the structured data associated with the image file:

Apple image search

Modern visual search is much more complicated, and still in its infancy.

What we’re beginning to see is the technology to take an image as the search query, so that the search engine interprets that image and sources the right results to match that visual query. It scours the image for shapes and colours and then finds similar objects to spit out in the search results. 

However, the current technology is still extremely limited. Machine learning needs to be able to replicate the brain’s image processing system which, given that we don’t understand it properly, is a big ask. Not only does a computer have to recognise an object, but it also needs to understand the variety of colours, sizes, shapes and patterns associated with it.

The most noteworthy visual search platforms include:


Ever since 2014, Pinterest users have been able to search for visually similar pins. With 90% of Pinner’s purchasing decisions being influenced by what they see on Pinterest, it’s a useful tool for brands.

However, going one step further, Pinterest launched its Pinterest Lens, which allows users to input an image as the basis of their search:

Pinterest Lens

Pinterest Lens connects your camera, lets you take a photo of an item, and then it’ll throw up pins that match that item as closely as possible.


In 2016, Bing launched its contribution to visual search with a new tool, Bing Visual Search. Originally a bit clunky (it required users to put a lasso around the item they wanted to search), the tool is now available on Android and iOS.

It allows you to take a picture of an item, or upload it from your photos, and then search for further information. The search results throw up related images which you can click on for more about the item.

Bing visual search


The latest version of Google Lens was launched in May 2018 and offers a simple, camera-based search function that’s native to Android devices. It can also be downloaded via Google Photos in iOS.

Similar to Bing and Pinterest, the app links with your camera to allow you to take a picture of an item and then search. You’re then asked to confirm your choice. The results will display items related to your search. Google have also updated the app and mobile results to include badges so that results will be given an icon in the bottom left to distinguish them as a product or a recipe.

Google Lens

Brands using visual search

Companies already using visual search are, predictably, fashion retailers. Most notable are Asos and Target (available in North America only).

ASOS’s Style Match Tool is available on its mobile app. It operates in a similar way to Pinterest in that it lets you take a picture of an item and uses that to search for products. Its primary benefit is for those who see a style icon or a friend wearing something they like, they can take a photo in that moment and search ASOS for similar items.

ASOS Style Match

How SMEs can start to adopt visual search

Get your images featured

Your first port of call is to optimise your images with structured markup. This means adding the right information to your images to give them the best possible chance of being found in visual search.

1. Implement your schema markup

For a product you’ll need to implement the “product” badge, for recipe, it’s the “recipe badge”.

You can also try to appear for “similar items” and “related items” by adding product markup to host pages including information like name, image, price, currency and availability.

For more on implementing the schema markup, see the Google support pages.

2. Confirm your markup

Once you’ve implemented your schema markup, you’ll need to validate it, by running some URLs through the Google structured data testing tool.

3. Wait for it to bed in

You may need to wait for up to a week for your site to be crawled and for any changes to show up in search, and it’s not guaranteed that changes will go through at all.

4. Test with a query

After a week or so, check that your images have been indexed by running a search on your site. If you don’t see any badges, then something’s gone wrong and you’ll have to go back to validating your schema.

Focus on visual quality

Aside from structured markup, the images you choose and how you treat them can massively impact on whether they’ll appear in search.

Stock images are increasingly frowned upon when it comes to sourcing visuals. There’s so much technology available for high-quality images to be produced that stock photos look unnatural by comparison, and therefore don’t achieve much traffic.

High-quality natural images will allow you to demonstrate your brand personality and so will get you decent clicks. Make sure you’re taking photos on a decent camera, and use online tools to make the best of them.

Canva and Pixlr (free), or Photoshop (paid) are good tools for doctoring images to make them online-ready.

Save your image file in the right size. If it’s too big it might slow down your site speed. Essentially your image size needs to be no bigger than the width of your blog. Squarespace suggests image sizes of around 1500 and 2500 pixels.

Finally, make sure your image is optimised to include keyword-rich image-alt tags, image file names, and descriptive captions.

For more on optimising images for SEO, read this Moz guide.

Build a visual search capability in your app/site

Although it might be a bit premature – and therefore expensive – for most SMEs to do this, 8% of companies have already built visual search into their web inventory.

As the software gets cheaper and integrations more readily available, this number will grow.

For most brands, it’s a case of watching this space.

Add an image sitemap

Publishing an image sitemap will give search engines more chance to find your images and have them displayed in search results.

To create an image sitemap, take a look at Google’s guide which takes you through the process here. 

Or, if you have a WordPress site, you can download the Google XML Sitemap for Images plugin, which will gather your images from your media library.

Use Pinterest to promote your products

With 600 million Pinterest Lens searches every month, the best way to get into visual search is by promoting your products here.

Its visual search technology and image recognition are probably the best available. Given the sheer numbers of pinners using the app to inspire their purchasing, having your product ad show up next to similar products is bound to bring you a good ROI.


The future of visual search for marketing is unclear, although as AI advances progress, possibilities for visual search will start to become clearer.

It’s unlikely that image search will ever replace text-based search. But it is likely that in the future both text and visual search will have to run alongside each other.

What is also likely is that visual search will be a big deal for the retail industry, which has, for a long time, been reliant on visual processing and image recognition.

But SMEs can start benefiting too. Savvy marketers can start to ensure that they’re making the best of their images by:

  • Optimising images for search.
  • Adding structured data (including badges) to distinguish them from other images.
  • Submitting image sitemaps to get images found online.
  • Creating ads on Pinterest that will be displayed when products are searched.