Vector Image Search with Azure AI

Hi Alexander,

Image Tags are attributes manually assigned by humans to describe or categorise an image.

ExtractImageTags is an internal method from the Azure Vision API that automatically analyzes images and generates tags. In other words, instead of humans analysing and assigning tags (attributes), Azure Vision API does this automatically for us, based on the content it detects in the image.

Hope this helps.

TJ Gokken Is it possible, instead of base dictionary labeled images, get centers of common image clusters from API? Because if you try to match some distant from your dictionary images (image of a cat when your dictionary contains only oceans/forests) you will get very confusing results. Or is it possible to get at least somehow unified distances between detected tags (cat and tiger are more similar than cat and toaster)?

Alexander Kolobov Great question. Yes, it is possible and might actually be a better way to match images. However, the focus of this article was using Azure AI Services. That's why we relied on the image tags generated by Azure Vision API.

At the moment, Azure Vision API does not support clustering of similar images. You can get unified distances between similar tags, but that would require additional processing on top of Azure's services.

Another point is, we are working with a very small set of images as our dataset. If our dataset was 5,000 or 50,000 images instead of 5, then the tags generated by Azure Vision API become far more effective.

Hope this helps.

TJ Gokken It is also possible to use Azure Functions that runs Python code to calculate tag similarities using a library such as Word2Vec. This allows more advanced similarity calculations between tags.