Monday, November 10, 2014

How to: Make sure User Profile properties are free/full-text searchable

You get a requirement to add a new property called FavoriteColor to the user profile application, which you want searchable on the people search page. You create it, you fill it in, you wait for a crawl so that the crawled property People:FavoriteColor shows up, you create a new managed property named FavoriteColor, map your crawled property to it, fire off a full crawl for people, head over to people search page and type your favorite color.

Nothing shows!

However, if you type FavoriteColor:color, you get results.

I’ve gotten this question four or five times now, so thought I’d write it up.

The solution is as hard as it is easy. When you create a new managed property, it is by default added to the Default full-text index, not the PeopleIdx full-text index. The full text-index is what is used when you perform a free text query (as apposed to a property query). When querying for people you have to make sure your managed property is in the PeopleIdx.

On the managed property settings page, click the  Advanced Searchable Settings button, and pick PeopleIdx from the dropdown.

image

Next up perform a new full-crawl of the UPA, and it should all work!

If you are curious what the FullTextIndex1/2/3 are, just stay away from them and don’t give it a second thought. They could in theory be useful, but when moving from FAST Search for SharePoint to 2013, some query logic was forgotten, so they are all but useless (known issue by the product team, and a ticket has been filed and archived). I might do a write up on that later – but not too many would be interested in it.

12 comments:

  1. I am very curious about FullTextIndex1/2/3

    ReplyDelete
    Replies
    1. You should be, but just forget about it :) I might blog on it but short story is they are unusable due to defects in the query API's.

      Delete
    2. And there is no way to create my own full text index to map properties to and search against?

      Delete
    3. Well..you certainly can. But you have no way to query it if you want ranked results. Ticked has been filed with MS, but suspect nothing will happen until critical mass ask for this....not that I know what that would be. (and it was filed once doing an internal MS project)

      Delete
  2. Hi Mikael, I hope you still read comments to your old(er) blog posts, because I could really use your help on this one:
    Hi Mikael, I hope you still read and reply to comments to your old(er) blog posts, because I could really use your help on this one.
    I've got similar issues like what your post is about. It's not exactly the same though, because I think I've got everything set up correctly and it works partially.
    I've got a couple of custom user profile properties that I want added to the FTI. I've created custom managed props, set them to Searchable, assigned them to PeopleIdx and set the Context ID to ID's used by default SP managed properties I want to match on relevancy. The problem I face is that I get no results on a search term when I use either ranking model "People Search name ranking model" or "People Search name social distance ranking model", but I do get results when using one of the other 4 People Search ranking models (e.g. "People Search application ranking model").
    Some more background info:
    - It's not that I don't get any results when using one of the People Search name ranking models, it's just that my managed properties appear to be ignored in the FTI.
    - When using one of the custom managed properties directly in a query (e.g. MyProperty:SharePoint), I do get results even in the "name ranking models".
    - I've tried various Context ID's but to no avail. I've tried 9 (because I want to use the AccountName context weight), 1 and 0 for testing purposes.
    - I see this behavior in all our environments (dev, test, acceptance, etc)
    I really hope you can point me in the right direction.
    Mark

    ReplyDelete
    Replies
    1. I don't remember exactly what those models look like, but most likely it has a limit on which fields are included in the FTI.

      If all you need is to make a profile property available in free text, the easiest approach is to map your custom people crawled properties to the managed property called ContentsHidden on the SSA or tenant level. Then re-index your profiles and they should be searchable. But it won't impact ranking any.

      So you really need to export the xml for the profiles and see how it's built - and I don't have any on-prem environments at hand, as I hardly never do on-prem these days :)

      Feel free to e-mail me the xml if you cannot figure it out and I'll check it.

      Delete
  3. Hi Mikael,
    Thanks for the super fast reply. I really appreciate that.
    We have various user profile properties that we want to include in the FTI but not every property should have the same weight. That's why I didn't use ContentsHidden.
    I haven't updated any of the OOTB ranking models. I just looked at the default managed properties inside the RankingModelXml (obtained through powershell) and looked up their Context ID's. I then used those same Context ID's in my custom properties to match the weight of the default ones.
    Any idea why the People Search ranking models behave so differently? I would understand different order of results due to different relevance determination, I don't get why 2 don't return any results and 4 others do. What could cause this?
    Thanks very much for your time.
    Mark

    ReplyDelete
    Replies
    1. As I wrote, could be that those two rank models have a different FTI component. Have you compared them?

      Delete
    2. Yes I have, but I don't really know what to look for. I looked at the BM25Main ranking feature of all People Search ranking models and although they're different, I can't find something that could be limiting the FTI size. The (default) "People Search application ranking model" has 9 default properties defined and weighted. This one appears to be working. The "People Search expertise ranking model" has only 1 property defined. This one worked ok as well. The "People Search name ranking model" has 7 default properties defined. I've also compared other properties of the ranking model but I can't find why it would work with one but not the other. I've read numerous posts about the Context ID matter, including your excellent post (http://www.techmikael.com/2015/01/how-to-boost-metadata-in-sharepoint.html) but somehow I can't seem to get it fully under control.

      Delete
    3. Hmm... hard to troubleshoot this without an environment. Which properties are in the name model, and are your custom mp's in any of the context levels for either of these?

      Delete
    4. Ok, after another day of frustrations I came to the conclusion that something doesn't work as expected with the default model "People Search name ranking model".
      I've ruled all my previous trials and errors out by recreating the Search service application. I then crawled my set of user profiles. Then I created a new managed property for one of the custom profile fields "MyCourses", set it to "Searchable", "PeopleIdx" and "Context 1". Then I performed a new full crawl.
      After this, when searching for "PRINCE2", I get quite some results when using the default model "People Search application ranking model", but zero on the default model "People Search name social distance ranking model".
      I then created a custom ranking model using PowerShell. This one has a custom name, description and ID but exactly the same RankingModelXml as the (broken?) "People Search name social distance ranking model".
      Using this new ranking model, searching for "PRINCE2", I suddenly get the same number results as "People Search application ranking model".
      I don't really get what the underlying problem is, but for now at least I can continue...
      If anything comes to mind that might help finding the root cause, please let me know. If I find anything, I'll put it here as well.

      Delete
    5. That's just weird. Do you also get different weights/rank using your new model? And guess you read this one http://www.techmikael.com/2015/11/people-search-ranking-for-dummies.html and this one https://blogs.msdn.microsoft.com/ericdixon/2015/06/03/sharepoint-2013-search-people-search-why-are-my-results-so-bad-understanding-relevancy-the-rank-model-full-text-index-fuzzy-matching-and-social-distance/

      Delete