We all want to deliver an exceptional search experience to our customers. Getting your users to the right search results doesn’t happen by chance. It requires continuous efforts to improve their experience. But, how do we measure improvement? A great scientist and engineer, Lord Kelvin, said that if you cannot measure it, you cannot improve it. In this post, we will address the questions regarding what to and how to determine search effectiveness. Once we get a firm understanding of how to measure search effectiveness, we can proceed further to explore the ways of improving it.
First of all, let’s begin with understanding metrics that help in determining search relevance.
3 Key Metrics to Measure Site Search Relevance
#1 Precision and Recall
For a search engine to be called as efficient, it is supposed to return only the relevant results and filter out non-relevant results. Two metrics that can tell how efficient your search engine is precision and recall. Precision is the number of results that are relevant, and recall is the number of relevant results that are retrieved. In other words, when recall is 1 it means ‘all’ the relevant results are returned, and when precision is 1 it means ‘only’ relevant results are returned. However, in the real world, neither of these two quantities are 1 at the same time. But, when you integrate Wizzy, a smart Magento search extension, you can bring results closer to this number.
#2 Search Results Position
Your site search may be showing relevant results, but something more important than relevancy is the position of the results. Users pay attention to the top results. According to a research, nearly 90% of users don’t look past the first page, and over 50% of users don’t look past the third page. Thus, precision matters for top ranked results on your page. We can tweak the precision metric to refine the top positions. One metric that can be used to weigh the relevance of each result based on the position algorithm is Discounted Cumulative Gain (DCG). It is particularly used when the relevance isn’t binary. DCG represents the gradation of relevance across completely relevant and irrelevant results.
#3 Search Exit Rate
Another good way to judge your site search efficiency is by measuring the exit-rate of the search results page. The exit rate of search results should be on par or close to your website’s average bounce rate. The search results page should have the lowest exit rates as possible. This metric shows the users who are searching with an intention to buy, and may exit your site because they couldn’t find relevant results. However, understanding your user behavior can further help you in measuring your search efficiency and improve it.
Although search relevance is often referred to helping people discover what they are looking for, but that is only a part of the picture. Even if your search results are well-tuned to display the most relevant results, it may not help you improve conversions if the search is running slow or delivering poor experience. When we talk about making improvements to your site search and how to boost e-commerce sales, it is necessary to improve the overall quality of the search. And, it all boils down to the combination of three main areas: user experience, search relevance and search performance. Let us dig into these factors and understand why they matter.
3 Factors that Contribute in Improving Search Quality
#1 User Search Experience
User search experience comprises of the functional aspects of how people engage with your search in order to find the products they are looking for. This includes the natural language processing feature, intelligent autocomplete, layout of the search results, spelling corrections, indexing, filters and synonyms. Moreover, the spellings and grammar of the content are also important. If they are displayed incorrectly, it may create a negative impression of your store.
In order to improve the user experience with your search, it is crucial to understand what your customers expect the search to display. You cannot improve experience if the improvement is not measurable with data. To do so, you can capture metrics for user experience through click tracking, A/B testing, heat maps of UI interaction, session length and so on. Capturing these metrics can help you get an understanding of how customers are interacting with your products and whether the experience is positive. When you are able to get a deep insight into these metrics, you can plan your strategy better and see which areas are performing well and which areas need improvement.
#2 Search Relevance
Search relevance refers to the issue when your customers are not able to find the product or information they are looking for. We define relevance as a series of quantities that measure whether appropriate results are displayed for a particular query, usually at the top of the page. Measuring the precision, recall and other such metrics, we can determine the relevancy of the search results.
These metrics can be measured by shooting more search queries. If adequate queries are being searched, we can cover more interactions that happen with your search. This is much kind of software testing, where the more tests you run, the less likely your customers will experience an issue. Moreover, it is crucial to measure how your users are interacting with your search in order to check if relevance is performing as expected.
#3 Search Performance
Optimum performance is essential to ensure that your customers enjoy browsing your site. Users today have very little or no patience when it comes to waiting for the search results to appear. Even though the relevancy is high and user experience workflows are at its best, customers may tend to leave your site if it takes too much time to load.
Managing KPIs for response times and page load speed will help in many cases keeping your customers satisfied with your search. Even if your relevance is not working at its best, it may still make your customers happy to see search results instantly.
However, if your search is still not meeting your customers’ expectations, you need a high level of diagnosis of issues. If you are not sure what is causing your prospects to turn away from your site, we have provided an insight into some common problems that may help you address search quality issues and rectify them.
Easy Tips to Solve Common Problems
Customers are searching, but they are not clicking on your products
Although you have a solid relevance plan in place, chances are the customers don’t click on your products. In that case, gather a wide range of search queries, and observe what results are returned for a particular query. Use metrics like recall and precision to improve your relevance.
Relevance is working well, but customers are still not clicking
If the customers are still not clicking even though relevance is working fine, then check whether the relevance coverage is high enough and if you are shooting the right queries. Go through the search queries your users are searching that don’t bring clicks and work your way up to improve them. Furthermore, check whether test data are in accordance with your customers’ expectation.
Some queries are returning very few or no results at all
This is usually the recall problem, which happens when the search is not optimized efficiently for synonyms. Integrate your search with natural processing language powered e-commerce search solution, and note down the list of common terms and synonyms. Add the words to the search engine dictionary and update it regularly, so to ensure that it returns the most relevant possible search results.
Conclusion
Measuring overall search quality is pretty difficult. However, we can assign tasks to users and measure it by seeing how efficiently they are able to accomplish those tasks. Users can face many problems while using search, so it is essential to look at the whole picture. Evaluation should be at the heart of the entire process. By broadening your investigations, you can explore new horizons that may help you take your search to the next level.
