Wednesday, March 12, 2014

Debugging Managed Properties using SharePoint Query Tool v2

The SharePoint Conference 2014 in Las Vegas March 3-6 had many great sessions on working with Display Templates and search. One of the tricky part at times when working with Display Templates and search results is making sure your managed properties have the data you expect, or simply seeing what data is available for an item.

To help with this, Matt King had a post on using the undocumented ManagedProperties managed property to list all available properties for an item.

When using the SharePoint 2013 Query Tool you enter the refinement filter below, which will return all the properties as a long refinement list.

managedproperties(filter=600/0/*)

image
Basically, a managed property exists which contains the names of all the managed properties for an item, within the item itself. The property is not retrievable (so you can see the content directly per item), but it is refinable, and via an extra call you can get the data back. This means that if an item has the following managed properties:
  • Author
  • Path
  • Write
then ManagedProperties will contain those three values “Author,Path,Write” as a multi-value refiner.

When creating my demos for SPC14 I noticed that this undocumented feature stopped working on some of my SPO tenants, and for new ones it didn’t work at all. It didn’t crash or anything, but no data was returned. Something had changed!

Also during the time the feature was working and the time it stopped working in SPO, Maximilan Melcher did some great work of implementing a “view all properties” feature into the Search Query Tool. The feature is so useful in my opinion that I decided to figure out what had happened and if there was a solution as I didn’t want to throw the feature out of the tool.

The feature is available in v2 of the Search Query Tool, and can be enabled as mentioned in my release post of the tool).

In the Search Query Tool you now have an option to view all the properties per item.
image

The image below show how this can look for an item, and gives you a full overview of which managed properties are available for your item to retrieve.



I pinged Microsoft on the issue, and lo and behold, something had indeed changed. The ManagedProperties property was originally created for use with cross-site publishing and catalogs, and is used internally with this feature. It was never meant for debugging purposes, however useful it may be.  As most libraries and list are not used as catalogs, populating this refiner creates unneeded resource strain on the search index, and the logic was changed to only populate it for catalog items where it’s needed.

The change has _not_ yet been made to 2013 on-premises as of SP1, but I suspect it will with a later update.

The change makes perfect sense from a performance perspective, but for debugging, it was and still is a very nice feature to have around. Turning on cross-site publishing on all site collections and then publishing all lists and libraries as a catalog didn’t seem like a very good approach, but it was enough of a hint to figure out how it all was tied together.

image

When you turn on Cross-Site Collection publishing on a site collection you enable an option on your lists for Catalog Settings.

image

Inside the settings page you check off to enable the list/library as a catalog.

image

What happens when you enable a list as a catalog is a couple of things. First an event receiver is attached to the list to handle the cross-site publishing on item updates. Secondly three properties are added to the property bag of the RootFolder for the library. Two of these three properties are the interesting ones,
  • vti_indexedpropertykeys
  • IsPublishingCatalog.
The combination of these two properties is what enables the crawler in SharePoint On-Line to populate the ManagedProperties managed property.

This means you can populate ManagedProperties for a list without attaching the event receiver by added the following to the RootFolder propertybag.

var folder = list.RootFolder;
var props = folder.Properties;
props["vti_indexedpropertykeys"] = "UAB1AGIAbABpAHMAaABpAG4AZwBDAGEAdABhAGwAbwBnAFMAZQB0AHQAaQBuAGcAcwA=|SQBzAFAAdQBiAGwAaQBzAGgAaQBuAGcAQwBhAHQAYQBsAG8AZwA=|";
props["IsPublishingCatalog"] = "True";
folder.Update();

If you’re curious on the first value, when you base64 decode it, it reads: PublishingCatalogSettings|IsPublishingCatalog

My guess is that this property is read first, and then uses the value inside it to figure out what property it should read the true/false value from.

If you want to enable population of the ManagedProperties managed property for all your data, you can iterate all your site collections, all sites and all libraries, and set these two keys/values in the root folder property bags. On the next full crawl of the lists, the data is added to the search index for your pleasure.

I have updated my SPO re-indexing script at https://github.com/wobba/SPO-Trigger-Reindex to include this feature with an option to enable or disable it. Happy debugging!

Note: The ManagedProperties feature should only be used for debugging purposes, and you should consider enabling it per list/library on a need to use basis only. The ManagedProperties refiner can get extremely large and will consume unnecessary memory which may impact query performance.