Friday, April 7, 2023

There are still new things to learn from the SharePoint Search API I won't share. I will NOT!

…I will, just a tad bit late :)

This was a tweet I made Friday October 21st once I understood the root cause of an API issue which has popped up in the last months. The issue affected PnP Modern Search web parts when query rules were enabled, and also the Search Query Tool with default settings.

I thought it was a weird API issue which had been introduced as part of an ongoing service upgrade for Microsoft Search lately, but turns out everything was working as expected – except the expected part hasn’t been expected for the past many years. And a big thank you to engineers at Microsoft helping me understand what the root cause is.

On the API side it manifests itself as follows with the following simple API REST query executed in the SharePoint Search Query Tool where you only get 2 main results where you would expect 10.

https://tenant.sharepoint.com/_api/search/query?querytext='mikael'&rowlimit=10

image

Where are the rest of my 10 results? Well, they happen to be located in the Secondary Results, a place I never looked.

image

I’ll explain the behavior, and it is actually correct (sort of), and I will explain why this happens now in 2022.

A trip down memory lane

When SharePoint 2013 was released Microsoft released the feature of query rules, which allowed to bring in result blocks into the search results as seen below.

The below screenshot triggers the rule “People Name in SharePoint” which bring in two result blocks.

image

And the query rule definition

image

The definition above says to always place people on top, and possible show documents authored by the person as a block within the results, or interleaved.

The thing is that the logic/setting to have it ranked has changed.

Todays logic – changed in October 2021
image

The old logic

image

The old logic introduced with SharePoint 2013 would start the block high up, and if results in the block was not clicked it would move down the page, and eventually off the page, which is what has happened for most customers over the years.

The new logic now introduced will ALWAYS interleave the block on the page 1 results, never moving it off the page.

So how does this affect the API?

By default an API query will invoke query rules unless explicitly turned off, e.g. the above query https://tenant.sharepoint.com/_api/search/query?querytext='mikael'&rowlimit=10

As Modern Search was introduced quite some time back this has greatly reducing the use of the classic search center. This means that people haven’t clicked results in result blocks from quite some time, no clicks are recorded and the block moved off the page – never appearing in API queries.

Now as the logic has changed, the blocks come back, which is not a bug, but maybe not expected.

Together with query rules there is another API setting available, one I have never thought about, but it’s been there all along. “Enable Interleaving” which by default is set to true documented at https://learn.microsoft.com/en-us/previous-versions/office/sharepoint-csom/jj262234(v=office.15).

Of course, if you don’t need query rules for your scenario you should always disable them on the API call. Problem solved!

Then again, using the PnP modern search web parts a common scenario is to use promoted results, and thus you need query rules enabled. Which leads to queries on people names triggering the original “People names rules” causing interleaving to happen and the results split into primary and secondary result tables in the response.

The solution then is to set EnableInterleaving=false.

Changing the query to https://contoso.sharepoint.com/_api/search/query?querytext='mikael'&enableinterleaving=false&rowlimit=10 ensure 10 results as expected in the primary result set.

image

I have released a fix to the Query Tool which by default will disable interleaving, or you can set it yourself.

https://github.com/pnp/PnP-Tools/releases 

And I have made the same fix to the PnP Modern Search Web Parts v4.8, and any interleaving should be done manually at the template level if strictly needed.