Friday, June 13, 2014

S15E06: Query Rules–Rewrite and paraphrase

Want the book? Go get it!

This is episode six in the series “SharePoint Search Queries Explained - The Series”. See the intro post for links to all episodes.

image

Two key concepts in search are recall and  precision. Recall refers to the number of results a search engine returns for a specific query, and precision refers
 
to how precisely the results match the search user’s intent. You must make a compromise between recall and precision. If recall is too large, you essentially flood the result set with noise, and precision suffers. On the other hand, when recall is too small, you run the risk of missing useful results, so the item(s) the user is looking for may not appear in the result set; again, precision suffers.

And this is where Query Rules play a part in SharePoint. By looking at the queries coming in, you have the option to change them to better match your content, and thus improve on recall and precision.
By default, recall in SharePoint search is expanded using a multitude of linguistic features to make sure variants of the terms entered are included. For example a search for test would also include tests, and vice versa.

Example 1 – improve recall

Say in your domain, the term test, also means exam and certification. Then you could use a query rule to expand on this (In SharePoint on-premises this could also be achieved using a thesaurus using PowerShell). The image below will do a two way synonym expansion, where it will match on either test, exam or certification, and expand with the same terms.

image
In the above sample I chose to use the Advanced Query Text Match condition, where I checked entire query, start and end matching. This means that the term test has to either be at the start or end of the query, or be by itself. In the list below the first three will match, while the last will not.
  • sharepoint test
  • test sharepoint
  • test
  • lets test sharepoint
If you are on-premises you have the option of writing regular expressions, and using regular expressions you can do about as advanced matches as you want.

Sample 2 – improve precision

Another rewriting sample can be to turn natural language into valid KQL syntax. Lets say a user types in: sales report last month, then you could turn that into the following rule.

image

First I decide to trigger on last month at the end of the query. Then I check that the trigger term is assigned to the {actionTerms} variable and the rest of my query, sales report, to {subjectTerms}. In the query rewrite I then use {subjectTerms} and append a filter on the write property using one of the special date filters mentioned in KQL – The Basics, to limit results to those of last month only. The ending query would look like: sales report write:”last month”.

Summary

In order to provide better end-results for your users you can start to examine your search logs for common queries. Then apply your domain knowledge to those queries and add query rules which change those queries into what the user actually meant. This is hard for a computer to do,  but as someone who knows the content you can apply your own smartness to help out your users.
If you have other examples, feel free to share them in the comment field.