Wednesday, March 12, 2014

Debugging Managed Properties using SharePoint Query Tool v2

The SharePoint Conference 2014 in Las Vegas March 3-6 had many great sessions on working with Display Templates and search. One of the tricky part at times when working with Display Templates and search results is making sure your managed properties have the data you expect, or simply seeing what data is available for an item.

To help with this, Matt King had a post on using the undocumented ManagedProperties managed property to list all available properties for an item.

When using the SharePoint 2013 Query Tool you enter the refinement filter below, which will return all the properties as a long refinement list.


Basically, a managed property exists which contains the names of all the managed properties for an item, within the item itself. The property is not retrievable (so you can see the content directly per item), but it is refinable, and via an extra call you can get the data back. This means that if an item has the following managed properties:
  • Author
  • Path
  • Write
then ManagedProperties will contain those three values “Author,Path,Write” as a multi-value refiner.

When creating my demos for SPC14 I noticed that this undocumented feature stopped working on some of my SPO tenants, and for new ones it didn’t work at all. It didn’t crash or anything, but no data was returned. Something had changed!

Also during the time the feature was working and the time it stopped working in SPO, Maximilan Melcher did some great work of implementing a “view all properties” feature into the Search Query Tool. The feature is so useful in my opinion that I decided to figure out what had happened and if there was a solution as I didn’t want to throw the feature out of the tool.

The feature is available in v2 of the Search Query Tool, and can be enabled as mentioned in my release post of the tool).

In the Search Query Tool you now have an option to view all the properties per item.

The image below show how this can look for an item, and gives you a full overview of which managed properties are available for your item to retrieve.

I pinged Microsoft on the issue, and lo and behold, something had indeed changed. The ManagedProperties property was originally created for use with cross-site publishing and catalogs, and is used internally with this feature. It was never meant for debugging purposes, however useful it may be.  As most libraries and list are not used as catalogs, populating this refiner creates unneeded resource strain on the search index, and the logic was changed to only populate it for catalog items where it’s needed.

The change has _not_ yet been made to 2013 on-premises as of SP1, but I suspect it will with a later update.

The change makes perfect sense from a performance perspective, but for debugging, it was and still is a very nice feature to have around. Turning on cross-site publishing on all site collections and then publishing all lists and libraries as a catalog didn’t seem like a very good approach, but it was enough of a hint to figure out how it all was tied together.


When you turn on Cross-Site Collection publishing on a site collection you enable an option on your lists for Catalog Settings.


Inside the settings page you check off to enable the list/library as a catalog.


What happens when you enable a list as a catalog is a couple of things. First an event receiver is attached to the list to handle the cross-site publishing on item updates. Secondly three properties are added to the property bag of the RootFolder for the library. Two of these three properties are the interesting ones,
  • vti_indexedpropertykeys
  • IsPublishingCatalog.
The combination of these two properties is what enables the crawler in SharePoint On-Line to populate the ManagedProperties managed property.

This means you can populate ManagedProperties for a list without attaching the event receiver by added the following to the RootFolder propertybag.

var folder = list.RootFolder;
var props = folder.Properties;
props["IsPublishingCatalog"] = "True";

If you’re curious on the first value, when you base64 decode it, it reads: PublishingCatalogSettings|IsPublishingCatalog

My guess is that this property is read first, and then uses the value inside it to figure out what property it should read the true/false value from.

If you want to enable population of the ManagedProperties managed property for all your data, you can iterate all your site collections, all sites and all libraries, and set these two keys/values in the root folder property bags. On the next full crawl of the lists, the data is added to the search index for your pleasure.

I have updated my SPO re-indexing script at to include this feature with an option to enable or disable it. Happy debugging!

Note: The ManagedProperties feature should only be used for debugging purposes, and you should consider enabling it per list/library on a need to use basis only. The ManagedProperties refiner can get extremely large and will consume unnecessary memory which may impact query performance.


  1. I am not able to enter anything into the box next to the Run button. Is there something special that I need to do so that I can edit the address?

  2. Never mind, i just found the Connection section of the tool

  3. hi
    unbelievable, an Enterprise Version
    to search on my own managed Property

    what's the problem with Microsoft

    I loose 2 days to find an answer, that come from Norway


  4. How do I limit search to specific site collection ? In connection I have specified a site collction url but results in this tool are shown from all site collections

  5. Hi Mikael. Awesome tool! Having some difficulty to run the script to enable View all properties for my SPO tenant.

    Have used nuget to grab the latest CSOM package: CSOM.16.1.7018.1200

    Have updated the script to point to the location where the file "Microsoft.SharePoint.Client.dll" resides.

    However, I keep getting the following error:
    "assembly is built by a runtime newer than the currently loaded runtime"

    Could you please assist if possible?

    Thanks in advance

    1. Hi,
      I assume you are using the script from ? If you start a blank PowerShell which is not loading any PnP or other SP commandlets - the loading of DLL's should work fine without issues.

    2. Hi

      Thanks for getting back to me.

      Yes, have downloaded the scripts from that location.
      Also confirming that I'm loading a blank powershell
      "Windows PowerShell"

    3. Try uninstalling any SPO/SP CSOM packages you have, and re-install it - and even though it's a blank posh, you might have things in Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1 loading CSOM?

  6. Hi Mikael
    I have just 'discovered' the SP search query tool. Downloaded it, but have trouble signing in. I want to connect to a SP2013 site through VPN connection. My options are: Windows, SharePoint Online (App id), SharePoint Online (Web login), Forms based and ForeFront Gateway. Which option should I choose? I selected Forms based, entered my credentials but no 'Sign in' option appears. Only when I select ForeFront Gateway, the sign-in appear, I clicked it, is shows Successfully signed-in. But when I want to run my query I get: HTTP/1.1 401 Unauthorized

    1. Hi Hes,
      If you connect via VPN fist, then using Windows and specifying credential should be the right option I believe.

  7. Apologies - please ignore my previous comment - I just used the windows login credentials and ran the query - it worked :-)

  8. Hi Mikael -

    I tried to search a general keyword and also updated the row limit to 9999.
    Under primary results, it is restricting to 500. Could you help me to view the results >500

    1. You need to page thru. Max 500 per page.

    2. Could you please help me how to page thru in the tool(SharePoint Search Query Tool v2.5)

    3. If you page 1-500, then 501-1000, then set Start:501 RowLimit:500. You can ignore rows per page, as it seems to be ignored.

    4. Thanks Mikael, this worked. But,after updating the start value to 501 we need to click the "Run" again to get the results. Is there a way to page through without clicking the "Run" button for the second time

    5. I suggest looking at the Submit-PnPSearchQuery commandlet I added. It has built-in paging, just beware of memory usage if you have very many results.

  9. Thanks Mikael. Sorry I was out for all these days

  10. how can i use the Search Query tool to look for Site Titles and/or site urls?

    1. Include the managed properties SiteTitle and SPSiteURL.

  11. Hi Mikael, did SharePoint Online fix this whole problem since you wrote this article? Because I am trying to replicate the issue with the ManagedProperties MP not being filled and it does work for all the sites I tried so far.

    My site does not have cross-site publishing activated and neither my list, but using the Search Query Tool v2 I do get all the managed properties available for a specific item.

    I actually ended up on your article by reverse engineering the OOTB "Highlighted Content" WebPart because I wanted to know how they managed to make a "Managed Property Picker" in the WebPart's property pane. After looking at the network tab, I found the exact POST request that it uses to retrieve the managed properties and it is pretty much the same thing found in your article.

    That being said, I would be surprised that the issue you are mentioning is still valid in SPO, since their own OOTB WebPart uses it. All this is because I want to make a managed property picker in a custom WebPart I am building but the issue you are mentioning picked my curiosity :)

    Let me know!

    1. They might have changed things indeed, and I've seen some other undocumented calls as well to search up managed properties lately.

    2. Alright, I will go with the assumption that I am safe to go forward with my custom managed property picker as long as the default "Highlighted Content" WebPart uses one :P


  12. Will this query managedproperties(filter=600/0/*) work for SPO? I'm looking to see all managed/crawled properties of a file in sharepoint online.