Thursday, January 22, 2015

Crawl Time for SharePoint Online and 2013

SharePoint Online (at least old tenants) has this nice managed property called CrawlTime. This is however not available in SharePoint 2013 (if you’re not getting the property back in SPO, see the end of this article).

Crawl time is stored in a crawled property named 323 in the Internal category. By default this crawled property is not visible in the schema, but if you create it, you can use it.

On-premises 2013

The script below will create the crawled property needed for crawl time, create a managed property called CrawlTime, and then map them. Next time you crawl an item you can get the crawl time value on an item….. very useful for troubleshooting without having to go to the crawl log to find the time it was crawled. You can modify the script below if you want to sort and query on CrawlTime to make the managed property queryable and sortable.

$ssa = Get-SPEnterpriseSearchServiceApplication
# To be run in a SP Management Shell
$cp = New-SPEnterpriseSearchMetadataCrawledProperty -SearchApplication $ssa -PropSet 012357BD-1113-171D-1F25-292BB0B0B0B0 -Category Internal -VariantType 0 -Name 323 -IsNameEnum $true
$mp = New-SPEnterpriseSearchMetadataManagedProperty -SearchApplication $ssa -Name CrawlTime -Type 4 -Retrievable $true -FullTextQueriable $false
New-SPEnterpriseSearchMetadataMapping -SearchApplication $ssa -ManagedProperty $mp -CrawledProperty $cp

SharePoint Online

If you’re not getting back data for CrawlTime, you can perform the below steps:
  1. Download CrawlTimeSchema.xml from https://github.com/wobba/SearchConfiguration

  2. Navigate to https://admin.sharepoint.com/_layouts/15/searchadmin/TA_SearchAdministration.aspx

  3. Click Import Search Configuration

  4. Choose the schema file you downloaded

  5. Click Import
image


You will now have a managed property named CrawlTime of type Text with the last crawl date. The reason for text is that SPO doesn’t allow creating of new managed properties of type Date.

If you want to query or sort on the property, you can map the crawled property Internal:323 which is now available to for example RefinableDate00.

5 comments:

  1. Hello Mikael -
    I'm trying to import the CrawlTimeSchema.xml into my Sharepoint Online, as when I run your reindex-users script the crawltime comes back blank consistently. However, when I import the XML I get the following error:
    Name: CrawlTimeSchema
    Scope: SPSiteSubscription
    Status: Import Failed
    Notes: There was an error deserializing the object of type Microsoft.Office.Server.Search.Portability.SearchConfigurationSettings. For security reasons DTD is prohibited in this XML document. To enable DTD processing set the DtdProcessing property on XmlReaderSettings to Parse and pass the settings into XmlReader.Create method.

    Can you provide any guidance on this error? Thanks in advance.

    ReplyDelete
    Replies
    1. Map the cp Internal:323 to for example refinablestring00 or refinabledate00, and you should see data coming in.

      Delete
    2. Unfortunately my only available internal properties are 3, 105, and 107.

      Delete
    3. I created a new file which I tested at https://github.com/wobba/SearchConfiguration/blob/master/CrawlTime_SearchConfiguration.xml It maps it to RefinableString100 and set an alias to PzlCrawlTime Make sure the file is in UTF-8.

      Delete
  2. Works like a charm now. Thank you so much for this. Is there an easy way to get the last crawl time for a single document in an easy query through the browser?

    ReplyDelete