July 17, 2024

Can Google See

Can Google SeeCan Google See?

Machine Learning and Search Results

Computer learning or as some may also refer to it as – machine learning – is a fascinating topic. Beyond it being fascinating, be aware that the machines are learning fast. This “learning” capability is also being applied to search algorithms and this is good for searchers as it means we, as searchers, will get better results when we perform a search.

The “machine learning” occurs in two ways:

  1. Machines can “learn” (understand better) about what we are “really” seeking. This means they understand us better.
  2.  Machines can “learn” about the content available to satisfy a search. This is the content that website and blog owners create. These content creators (you may be one of them) also make an effort to optimize the content so that it will be found by search engines, categorized properly by the search engines and then outperform the competitors related to the topic of our content. Search algorithms play a major role in this activity.

As the search engine “machines” learn we, as searchers, can also execute more specific or more difficult searches and expect to get meaningful and valuable results. This is wonderful for people doing research on essentially any topic – whatever the topic is!

What this means for content marketers is that we must up our game. The top results will now be even more relevant as the search engines deliver ever more accurate results. Therefore, the top results will command the far majority of the clicks. With the better and ever improving search results if you are on the 2nd page you are invisible. If you are low on the 1st page (once a respectable position) you will be hard to find.

In a study, last year, by Advanced Web Ranking shows that fewer than 5% of searchers click on a result which was 5 or greater.

Click Through Rate versus Position in Search

This post is about image search and how good Google is at delivering results. There is a powerful message for marketers as images are a very important aspect of search performance for certain items and markets.

Here is an example – for a search based on an image search – of how good Google is now at understanding website content. I wanted to find a cute Shihpoo puppy and Google delivered the results in a very significant and amazing way.

Here was my search.

Google Image Search

I did a search for Shihpoo puppies using the Google image search function. As expected I get a wide range of results as Google wants to show a variety of content. I was satisfied with the results presented by Google.

Image Search Results for Shihpoo Puppies

Now it gets interesting.

I really wanted to search for black Shihpoo puppies. Of course, Google would have no way of knowing this until I better indicated what I really wanted.

With the results provided for the more general search for “shihpoo puppies”, I selected the image of the black puppy in the top row.

Image Search Refinement - Black Shihpoo Puppies

Here is the selected image with a more refined display – showing only black puppies on the right. I then clicked on the “view more” link.

Now all the puppies are black. However, there is even more to ponder.

Google Image Search - Refined for Black Puppy Search

They images have the following in common:

  • All black
  • Similar backgrounds
  • Similar head orientations

I believe that this is an excellent display of very good search technology. Many SEOs think that Google cannot see images. SEOs think that we need to name the image file in a manner that describes the image. We need to use alt text to provide the necessary clues to Google so that Google “knows” the content of the image. Perhaps there is more going on.

Let dig in deeper. Can Google really see?

Here are the file names and the alt text for the 1st five images across the top. These are the top image results returned by Google. Surely these images are fully optimized in terms of the text used so that they are fully described as a black Shihpoo puppy (image file name and alt text).

Image 1

Image Search Black Puppy 1File Name: Shih-Poo-Shihpoo.jpg

Alt Text: none


Image 2

Image Search Black Puppy 2File Name: Dexter-Shihpoo.jpg

Alt Text: none


Image 3

Image Search Black Puppy 3File Name: pup_1373383573.jpg

Alt Text: Shih-Poo Puppies For Sale In PA!


Image 4

Image Search Black Puppy 4File Name: pup_1373383501.jpg

Alt Text: Shih-Poo Puppies For Sale In PA!


Image 5

Image Search Black Puppy 5File Name: shih-poo-shih-tzu-toy-poodle-puppies-2-males-1-female-americanlisted_37979783.jpg

Alt Text: Shih-Poo (Shih-Tzu/Toy Poodle) Puppies 2 males 1 female


I will note that there was also a Twitter card for image 5 but the information was the same.

There was no schema markup code or rich snippets on any of these pages to help Google.

I also examined the image file name paths for each image and there is no information that would indicate a black color or even a puppy to say nothing of a Shihpoo Puppy in the path statement.

I repeated the exercise for brown puppies with the following results.

Can Google See - Image Search

Perhaps the results are not quite as good as for the black puppy (there is a white are some white and brown and very dark brown) results but the results are very good none-the-less. I suspect that brown is a bit more nuanced than black making it harder for the algorithm.

Here are the text-based details for the five images across the top for these search results.

Image 1

Image Search Brown Puppy 1File Name: pup_1381419191.jpg

Shih-Poo Puppies For Sale In PA!


Image 2

Image Search Brown Puppy 2File Name: pup_1365994335.jpg

Yorkie Poo Puppies For Sale In PA


This was on a page with perhaps 30 puppies of all different colors, sizes, backgrounds and head orientations.

Image 3

Image Search Brown Puppy 3File Name: ba200b9e9c809a1b1b7f36196336f405.jpg

Alt Text: none



This was on a Pinterest page with many other puppy images and no description and no tags.

Image 4

Image Search Brown Puppy 4File Name: cavachon-rare-apricot-tipped-and-ruby-puppies-53d8ce5d2a13d.JPG

Alt Text: none


This image was on a page with descriptive title tags, description tags and a significant amount of text content using the words such as ruby or apricot which does describe the color.

Image 5

Image Search Brown Puppy 5File Name: Rocky-Poo-Shi1.jpg

Alt Text: none


Note that I only did this using Google. My focus was on Google as it is my search engine of choice.

I conclude that there is something very significant going on – object detection within a photo is not depending on image files names and image alt text.

At the end of the day, it all comes down to your position in search. Image content understanding adds an extra layer of ranking signals that cannot be easily altered or used to “trick” the search engines. If it is an image of a black Shihpoo puppy, then it is an image of a black Shihpoo puppy.

An image of a black Shihpoo puppy will rank for a search for “black Shihpoo puppy” and not for a search for “white Shihpoo puppy”.

A page with many images of black Shihpoo puppies (or perhaps the same puppy but with multiple views) will be a page with more search engine “strength” compared to pages with a single image.

Whenever there is competition there is progress.

Google is always testing its algorithms. Outside of Google, other clever algorithm creation teams are involved in a major challenge. It is called the Large Scale Vision Recognition Challenge. Google is a major sponsor of this challenge along with Facebook.

The competition is organized by Stanford University, The University of Michigan and The University of North Carolina at Chapel Hill. With the involvement of Google, Facebook and these top universities you know that excellent results will be achieved.

There are multiple aspects to the challenge. For example, a part of the challenge is to compete in the ability to properly identify multiple objects in a single image. Submitted algorithms will be evaluated on their ability to detect specific objects in an image as shown below.

Google Image Search - Object Recognition

The challenge also involves properly identifying specific items in photos without making errors. Here are some specific images showing how an algorithm of this type identifies what is in the image along with other potential results (errors).

Google Image Search - Object Recognition - Hens

Google Image Search - Object Recognition - Hot Dog versus DogI did an image search just before publishing this post using Google and there were no non-dog result errors for the first 156 images returned. The 157th image was this image.

Perhaps we can understand the error, after all, we are humans and not machines so we are still better (at least for now) at properly identifying items in a photo.




I am confident that Google will continue to make improvements and refine the results. The approach may be to give objects and classes of objects a unique code (so, for instance, a jaguar – the animal – could never be mistaken with a Jaguar – the car) and the code is used to help the algorithm “learn” by providing it with a knowledge base against which to test its attempts.

Can Google See?

The conclusion is that Google can “see” images. It does not need to depend on image file names and the image alt text.

As content creators with a keen desire to be found in the search results. We need to be aware of and learn to leverage the new technologies being applied by the search engines.

Google and the other major search engines want to return the more accurate results and they are highly motivated to do so. We, as content creators, want our content to be found accurately so that it is presented to the ideal audience we are seeking to attract. We need to factor images and graphics into our search optimization strategy.