So how does it really work?
A common misunderstanding when it comes to image search is that the technology is based on detecting information in the image itself.
But most image search works as other search engines. The meta data of the image is indexed and stored in a large database and when a search query is performed the image search engine looks up the index, and queries are matched with the stored information. The results are presented in order of relevancy. The usefulness of an image search engine depends on the relevance of the results it returns, and the ranking algorithms are one of the keys to becoming a big player.
A limited range of visual content can be identified automatically e.g. faces, trees, sky, buildings, flowers, colours etc. This can be used to augment metadata in image search or more directly through the use of a dedicated retrieval language as by Imense. To extend this to larger picture, Google image search allows you to identify Primary colors used in the images. That would be like looking for a binary match for say “red color” in complete JPEG format.
Ok, Enough from me, lets hear it from experts, Google themselves!