Wednesday, October 9, 2013

Make sure your People Search is fuzzified

Topics covered in this post
  • Fuzzy name matching on people search
  • Setting default language of a search result web part
  • Using the &ql URL parameter to set query language
  • Using Reflector to figure out how it all works
If your users primary language setting in SharePoint is a minority language, this post is for you. If your primary language is one of the languages in the list further down, keep on reading as well to broaden your horizon.

Finding people is one of the most used search features in SharePoint, and spelling names is inherently hard as people choose just about all possible ways to spell their name.

As an example; my name is Mikael Svenson, where it’s more common to spell Mikael with ch instead of a k (Michael) and Svenson is most commonly spelled with two s’ in the middle (Svensson). This means a search for “Michael Svensson” should also match “Mikael Svenson”. This is where fuzzy name matching comes in.

Search in Norwegian – yields zero results - ql=1044

image

Search in English – returns a match -  ql=1033

image

Looking at the linguistic feature table over at TechNet you see the languages supporting fuzzy name matching are:
  • Dutch 1043  /2067
  • English 1033
  • French 1036
  • German 1031
  • Italian 1040
  • Japanese 1041
  • Polish 1045
  • Portuguese 1046 / 2070
  • Russian 1049 (not working) / 2073
  • Spanish 1034 (not working) 3082 / 9226
Seems most of the above languages work on my test case, but I’m not sure how the logic differs on each one. What I do know is that the English one is pretty good and works, while the Norwegian one is non-existent. This means if I use Norwegian versions of SharePoint and my operating system, my queries are most likely to be executed in a Norwegian context which disables fuzzy name matches.

The question you might ask yourself is: “How does SharePoint decide the language to use when executing a query on a search center result page?”
After some digging around I found the logic to be as follows in sorted order:
  1. Use a fallback language if present
  2. Use query language URL parameter if present (more on this later)
  3. Use the users preferred language if present (https://mysitehost/_layouts/15/EditProfile.aspx?Section=LanguageAndRegion&ShowAdvLang=1&UserSettingsProvider=dfb95e82-8132-404b-b693-25418fdac9b6)
  4. Use the browser language
The solution for me was setting the fallback language to English or 1033 in the result web part on the people search page. By default this property is not set.

What is the fallback language?

The fallback language name is a misnomer as it is in fact a language override parameter, and it’s a property of the DataProviderScriptWebPart. If you export the People Search Core Result web part from the peopleresults.aspx page you find the property well hidden in a JSON object in the web part’s DataProviderJSON property.

image

The property is by default –1, which means it’s not set. By changing the value to 1033, all queries will now be executed in an English context. In order to get activate the change you may either upload your changed .webpart file to the web part gallery and replace the web part on the people result page, or in an on-premises solution you can use the following PowerShell commands. What it does is checking out the page if needed, then reading the DataProviderJSON data, changing it, setting it back and committing the changes.
$web = Get-SPWeb http://intranet.contoso.com/search/
$fileName = "Pages/peopleresults.aspx" # replace "Pages" with your locale if necessary

# Modifying web part for people search core results
$page = $web.GetFile($fileName)
try{$page.CheckOut()}
catch{ Write-Host "Page already checked out" -ForegroundColor Yellow}

$wpm = $web.GetLimitedWebPartManager($fileName, [System.Web.UI.WebControls.WebParts.PersonalizationScope]::Shared)
$searchresultsWP = $wpm.WebParts | where Title -eq "People Search Core Results"

$dataProvider = ConvertFrom-Json $searchresultsWP.DataProviderJSON
$dataProvider.FallbackLanguage = 1033
$searchresultsWP.DataProviderJSON = ConvertTo-Json $dataProvider -Compress

$wpm.SaveChanges($searchresultsWP)
$page.CheckIn("Modified Search Core Results web part")
$page.Publish("Modified Search Core Results web part")
$web.Dispose()


How did I figure this out?


First I looked at the exported .webpart file and found the FallbackLanguage property which seems like something useful. Then I did an internet search on the property and found the parent object, DataProviderScriptWebPart. I then fired up Reflector to find what other code is using this property, which led me to Microsoft.Office.Server.Search.WebControls.ScriptApplicationManager which has an internal method called GetLanguage listed below:

internal int GetLanguage(QueryState qs, DataProviderScriptWebPart dp)
{
    int fallbackLanguage = -1;
    string str = string.Empty;
    if (((fallbackLanguage < 1) && (dp != null)) && (dp.FallbackLanguage > 0))
    {
        fallbackLanguage = dp.FallbackLanguage;
        str = " from fallback language...";
    }
    if ((fallbackLanguage < 1) && (qs.l > 0))
    {
        fallbackLanguage = qs.l;
        str = " from query state...";
    }
    if ((fallbackLanguage < 1) && (this.defaultUserPreferenceLanguage > 0))
    {
        fallbackLanguage = this.defaultUserPreferenceLanguage;
        str = " from default user preference language...";
    }
    if ((fallbackLanguage < 1) && (this.browserLangage > 0))
    {
        fallbackLanguage = this.browserLangage;
        str = " from browser language...";
    }
    ULS.SendTraceTag(0x1d846, ULSCat.msoulscat_SEARCH_Query, ULSTraceLevel.Verbose, "GetLanguage::Setting query languge to '{0}'" + str, new object[] { fallbackLanguage });
    return fallbackLanguage;
}

The code above checks also check on the QueryState object, and this leads us over to the….

Query Language parameter

Reflecting a bit more on where QueryState.l is being set, I was led over to the Microsoft.Office.Server.Search.WebControls.KeywordQueryReader class (which is an internal class) and the Initialize method which parses the URL query parameters. The interesting part being:

else if (this._QueryId == QueryId.Query1)
{
    param = "ql";
    if ((queryString[param] != null) && !int.TryParse(queryString[param], out this._QueryLanguageID))
    {
        this._QueryLanguageID = -1;
    }
}

This parameter was news to me, but could prove quite helpful in the future.

2 comments:

  1. Hi Mikael,

    To enable fuzzy logic for all custom search pages we have created the query rule for all other vertical (Copy the OOB same rule for other vertical with other name “People Name in Sharepoint Search”).
    The fuzzy name matching is working fine for people results page. But other pages it is not working as expected.

    I have posted the same issue in MSDN, Kindly suggest.

    https://social.technet.microsoft.com/Forums/office/en-US/fa6f99ad-35f6-4ade-8e12-53a9c36a9c6c/fuzzy-match-against-people-name-not-working-in-search-vertical?forum=sharepointsearch

    ReplyDelete
    Replies
    1. The fuzzy matching not working in other search Search vertical, I Any idea why it's causing issue?
      https://social.technet.microsoft.com/Forums/office/en-US/fa6f99ad-35f6-4ade-8e12-53a9c36a9c6c/fuzzy-match-against-people-name-not-working-in-search-vertical?forum=sharepointsearch

      Delete