Search Beyond the Words: AI Vector Search vs Traditional Text-based Search

---

Unlike traditional search, vector search understands concepts beyond plain text and keywords, allowing users to search for products through images efficiently.

Our online experience today relies heavily on search engines, which help us discover content and products on platforms like YouTube, Netflix, eBay, Etsy, etc.

However, traditional search engines, while widely used, have limitations. They rely solely on textual data and depend on exact or partial matches to the original search term, often leading to inefficient searches and inaccuracies.

So, how do we move up from text-based search? The answer is vector search! It’s an AI-based search that operates at a higher semantic level. Unlike traditional search, vector search understands concepts beyond plain text and keywords. It can also analyze and understand images, allowing users to search through images and visual content.

At Velebit AI, we excel in AI search technologies for e-commerce platforms and online marketplaces, which is why we often receive requests to improve search functionality on these platforms. This has motivated us to write this comparison blog between our AI vector search engine and Elasticsearch engine.

Our goal is to demonstrate which search engine produces better results with just a simple text query:

• traditional search through textual data with ElasticSearch or
• searching through images with Velebit AI vector search

Differences between vector search and traditional search

Vector search uses AI technology to understand the meaning and context of unstructured data, including text and images, by converting it into a numerical representation. It then identifies similar data using a vector similarity metric, with cosine similarity or L2 distance most commonly used.

In other words, when you search using natural language (text-based queries) on e-commerce platforms and online marketplaces, AI vector search delivers more relevant results by connecting natural language queries with the content of the images. This means users can find product and content images that match their search intent, even if the exact keywords aren’t in the description.

On the other hand, traditional search methods rely solely on text, often struggling to grasp the semantic nuances of user queries. They primarily depend on image metadata, such as filenames and alt text. This limits their ability to understand user queries and analyze visual content, often resulting in unsatisfactory search results.

To compare search performance on e-commerce platforms using natural language queries, we compare Elasticsearch, a text-based search engine, with AI vector search. The latter uses a multimodal machine learning model to map images and textual queries to a unified feature space, enabling more accurate results through vector search.

ElasticSearch vs Velebit AI vector search

Elasticsearch

Elasticsearch is a full-text search engine widely used in the industry. It’s a popular choice because of its powerful and intuitive API, which makes it easy to use on custom documents.

It works by breaking a textual query into tokens, processing those tokens, and finding documents where said tokens appear. It then ranks matching documents by looking at certain relevancy factors like the frequency of a token appearing inside the document and returns top-ranked documents.

When working with images, Elasticsearch matches input queries to the image metadata, like title and description, and returns the best matches.

AI-based search

Opposed to Elasticsearch, the AI model used in the AI search engine does not require image metadata at all. Rather it uses just the images themselves. The search uses a multimodal model that processes both image and text.

It works by mapping images and text into the same feature space in a way that semantically similar images and textual queries are mapped close together. So, a feature vector of the term “cat” would be close to feature vectors of various cat images and not very close to feature vectors of bicycle images.

A vector search can then be performed in this feature space. The model takes a textual query, calculates its feature vector, and retrieves images from the vector search database whose vectors are closest to that of the query.

How we compared the search approaches

To compare the results of the two search approaches, we found a public dataset containing images with textual descriptions of the images. We then searched this dataset using our two search engines. For a given query, Elasticsearch used only the textual descriptions of the images, while AI-based search used only the images to perform the search.

Search results were manually assessed to see which engine performed better on individual queries, and the engine’s score was calculated as a percentage of queries on which it outperformed the other engine.

Dataset used for comparison

The performance of the AI model and Elasticsearch was tested on a Kaggle dataset containing 44 thousand images of various clothing items. The images are catalog images and contain textual information about specific products. When comparing Elasticsearch, the main focus of the search was on the product display name, but other product fields like brand name, age group, gender, and color were used in the search as well.

To compare the performance, textual queries in the style of common e-commerce search queries were written and passed to the engines. The recommended images returned by the engines were then visually compared. Recommendations were assessed subjectively to determine which engine performed better on the given query.

Results comparison

For common fashion terms and simple queries describing basic properties of products like color or general design, both engines performed equally well and returned satisfying results. In some cases, the two engines recommended completely different images that equally matched the query. In other cases, search engines generally recommended the same images but in different order of relevance.

Query: blue men's shirt

AI Vector Search Results

ElasticSearch Results

For this and similar queries it is hard to say that one engine is better. Even though engines tend to recommend different images or rank the images differently, both engines’ recommendations are valid. Engines generally recommend images related to the query.

For more specific queries and longer, more detailed descriptions AI model proved to be better than Elasticsearch. For example, when given a query women sandals white straps, AI performs much better. While Elasticsearch recommended sandals, most of them did not have white straps. AI-based search, on the other hand, only recommended sandals with white straps.

Query: women sandals white straps

AI Vector Search Results

ElasticSearch Results

Other than more detailed queries, AI Search also proved better on queries with not enough details. For the query superhero shirt, AI recognized the semantic connection between superheroes and Superman and Batman, recommending shirts accordingly. Meanwhile, Elasticsearch recommended random images since none of the images in the database had superhero in the textual description.

Query: superhero shirt

AI Vector Search Results

ElasticSearch Results

The AI Search model also recognizes certain details in the images, like with the query flounce. While Elasticsearch struggles and does not return any results because none of the image descriptions contain the word flounce, the AI model returns several shirts with flounce decorations.

This is the power of the AI approach. Since it looks at the whole image, and an image is worth a thousand words, it knows more information about the image than what could be written in a reasonably sized textual description.

Query: flounce

AI Vector Search Results

The AI model can go even further, recognizing and reading text on objects. If the model is asked to retrieve t-shirt with “thursday” print, it will find a t-shirt with that printing, as seen below. It will also return t-shirts with similar prints, like the t-shirts with ‘sunday’ or ‘saturday’ written on.

When the model is unable to find any more similar t-shirts, likely due to the dataset limitations, it will return unconnected t-shirts with random prints, ranking them lower than the aforementioned matches.

Elasticsearch, on the other hand, did not find adequate images since the word ‘thursday’ was not in the description of any image and returned generic printed t-shirts.

Query: t-shirt with “thursday” print

AI Vector Search Results

ElasticSearch Results

In total, 35 queries were passed to the search engines. The results of the engine comparison are given in the table below. As we can see, the multimodal AI model surpasses ElasticSearch. The model can detect all the features listed in the image descriptions and much more.

Elasticsearch could be further improved with more detailed descriptions; however, real-world product listings are often non-descriptive, and adding more textual information seems unnecessary if the AI engine works adequately without any textual information.

Winner	AI Vector Search	Tied	ElasticSearch Text Search
Examples	77%	23%	0%

Benefits and drawbacks of AI search engine

These examples show several benefits of the AI model, but there are several more that are not shown. The main benefits are:

• The model can detect types of objects, object colors, and patterns.
• The model can detect and interpret text, images, and drawings on objects, as well as other smaller details, like flounce.
• The model can understand semantic connections between images and can recommend images that are not syntactically directly connected to a given query.
• The model is robust to spelling errors and works well with synonyms and alternate wording of queries.
• The model works on both catalog and non-catalog images.
• Unlike Elasticsearch, the model will always return some results. Even for gibberish queries, it will produce recommendations, albeit gibberish.

The model has one more important feature: its interpretability. Oftentimes, AI models feel like magic. They perform complex mappings to abstract high-dimensional spaces incomprehensible to humans who live in a three-dimensional space.

However, the model’s multimodality allows us to see firsthand which words are similar and will return similar images, which words the model interprets wrong or doesn’t know, which words are connected to a certain image, and more. This sheds light on the model’s computing process and allows us to examine its performance and find errors more easily.

On the other hand, there are also some drawbacks. Compared to searching through ElasticSearch, this system is more complex and consists of more components:

1. AI model - converts content (image, text, or a combination of both) to a vector
2. Vector database - stores a set of vectors

AI models are usually deployed on specialized instances with GPU cards that make them more efficient. These instances cost more, which increases the overall cost of production. More on deployment is in the next chapter.

Deployment comparison

In terms of deployment, Elasticsearch (or any other full-text search database) and AI-based search differ significantly. Elasticsearch is a search engine database specialized in text-based search. It breaks down documents into tokens, which are then stored in an inverted index along with references to their respective documents. This allows for a quick look-up of token locations that are used in searching.

In contrast, AI-based search requires not only a database for image feature vectors but also an AI model for vectorizing text and images. Thus, AI-based search contains an additional component (AI model) that is computationally heavy.

For production purposes, the best solution for storing and searching image feature vectors is a vector database. These databases are specialized for storing vectors and allow for quick retrieval of vectors closest to the query vector. Popular choices for vector databases include Milvus, Pinecone, and Chroma, but Elasticsearch can also be used since it supports vector search.

Use cases

Anywhere images are present, AI can be used in conjunction with traditional search engines to find results that textual search may have missed and enhance the user experience.

For use cases where verbose descriptions of the items or their detailed specification are provided, Elasticsearch or a similar search engine could be enough. Think about searching for cars: in the initial phase, you are more interested in the specs (like model type, age, power, etc.) than in the image. Of course, you also want to look at the pictures, but more to assess the car’s current state.

However, when detailed descriptions or specifications are not present, AI can be a good alternative. This is a typical case for e-commerce websites and online marketplaces, where other users tend to write unspecific product listing descriptions. Here, using AI alone or combined with textual search engines can significantly improve the user’s ability to find desired items.

E-commerce platforms & online marketplaces

AI vector search enables more accurate and intuitive product searches. Unlike traditional text-based search engines, AI vector search can understand images, allowing users to search for products based on visual attributes. 

For example, a marketplace user searching for a “Scandinavian-style oak wood TV stand” can easily get relevant options even if the product descriptions don’t explicitly mention the Scandinavian style or oak wood for a TV stand, improving the overall shopping experience and increasing the likelihood of finding the desired item quickly.

Fashion & apparel

When it comes to online fashion platforms, AI vector search can significantly help in product discovery based on natural language queries. If a user is searching for “bohemian summer maxi dress” they can easily find relevant options for long-tail keywords. This technology eliminates the need for users to rely only on textual descriptions, improving overall product search and discovery.

Artwork advanced search & design platforms

Artwork or design search platforms greatly benefit from AI vector search by enabling users to find relevant content with better accuracy. For example, a user searching for “modern abstract art for living room” can receive more precise results.

AI vector search can analyze images and understand design elements such as color schemes, patterns and styles beyond simple keywords, which helps users quickly find items they may not have the exact words to describe, making the search process more intuitive and efficient.

Real estate websites

Instead of relying only on text descriptions, AI vector search can analyze images of properties and understand features like architectural styles, room layouts, and interior design. This results in more precise matches for users searching for specific criteria, such as “modern kitchen” or “artsy living room.” It also enhances the user experience by saving time and ensuring that properties align closely with the preferences of potential buyers or renters.

Conclusion

AI-based vector search engines represent a significant advancement in search technology, offering improved accuracy and efficiency compared to traditional text-based search engines.

These advanced AI solutions excel at understanding natural language queries, recognizing visual content, and providing relevant results even without complete textual information, as demonstrated in our comparison.

As we continue to generate and share more visual content, the demand for effective image search technology will only increase. The combination of AI, machine learning and vector search creates a potent toolkit to meet this demand, enabling a new generation of image search capabilities that can understand visual content from text-based queries.

This solution will become increasingly accessible to businesses of all sizes, making the search experience more enjoyable for everyone. For e-commerce and online marketplaces particularly, integrating AI vector search is a way to go at the moment, as it offers substantial improvements in user engagement & product discovery, ultimately boosting conversion rates and sales.

If you want to learn more about Velebit AI’s e-commerce search & product discovery tools, make sure to check out our cost-effective and easy-to-integrate AI products - Visual Search, AI Recommender & Color Detection API! Get in touch with us if you need help on your AI implementation journey!