Thursday, August 11, 2011

Why you should use deep refiners with FAST for SharePoint

With FAST for SharePoint you have what is called deep refiners, compared to shallow refiners which the built-in search uses. The difference is that deep refiners guarantee an exact refiner count, and all possible refiner values for your search, while shallow will calculate the counts based on the first 50 hits (default setting which can be adjusted). In addition to deep refiners, FAST for SharePoint also supports shallow refiners.

The first image shows a query using FAST for SharePoint with deep refiners and the second image show the built-in SharePoint Search with shallow refiners.



With FAST for SharePoint deep refiners are stored in a separate data structure (mostly in memory), optimized for just that; returning the refiners. Think of it as a pre-calculated refiner tree, ready for the picking.

Shallow refiners on the other hand have to be manually constructed by the query server from the results returned. Returning a large result set, looking at the metadata and building the refiners, is both IO and CPU intensive, as it has to retrieve all metadata per item, and then build the refinement result based on that.

Of course, basing the refiners on 50 results is a speedy operation, but at the cost of the accuracy of deep refiners.

So, when you create a refiner with FAST for SharePoint, be sure to tick off the “Deep refiner” checkbox to take advantage of the refiner structure stored within the search index. You will not only get more accurate results, but in most cases they will also return even faster than shallow ones.

PS! There are cases when deep refiners will be slow, particularly if you have very many unique values within a refiner. Then again, if this is the case, you might want to re-think your search solution.