Monday, February 22, 2016

When query by content type name use SPContentType and not ContentType

If you are to query in SharePoint using search using the name of a content type, be sure to query using the managed property SPContentType and not ContentType. The latter contains extra data, and will usually fail you.

Use SPContentType=Document, and not ContentType=Document

Of course you can query using ContentTypeId instead if you have the id.

Tuesday, February 16, 2016

Why you should back up your term store in SharePoint Online

image
Short version, if you delete a term, you cannot easily restore it.

But it is possible! (solution at the end of the article)

Monday, February 15, 2016

Using SPServices to help populate fields in a SharePoint form

image

I know SPServices is old skool and so am I. We both go very well in hand with web services indeed, but we do enjoy the occasional re-write to new world of REST. Honestly!

I have a form where based on a lookup to another list I should populate, or copy values, over to some fields. Pretty simple task, which is not out of the box.

As I hate writing plumbing code I pinged Marc Anderson and found out that his SPServices library has a method named SPDisplayRelatedInfo. This method is meant to be attached to the drop-down tied to the look-up list, and will pull in related data and render it as an inline table.

(Image borrowed from https://spservices.codeplex.com)

In my case I wanted to populate the Site name, Department and Responsible person based on the item selected in the below drop-down.

image

Thursday, February 11, 2016

Search schema mapping level explained

In SharePoint search you can map a crawled property to a managed property at basically three different levels. On-premises farm have SSA, site collection and site, while SPO have Tenant (subscription), site collection and site.

Search Service Application / Tenant

This is the top level, and mappings done at this level are inherited down to all site collections and sub sites.

Mappings for content outside of a SharePoint site collection like user profiles, hybrid crawled data, bcs, file crawls etc HAS to be mapped at this level.

Site collection

Mappings at a particular site collection will have effect for content stored on that site collection only, and only for searches executed in the context of the site collection. The last part is important. If you search center or search page is residing outside of the site collection where you did the mapping, then the mapping is not applicable. Any mapping is also inherited down to sub sites.

Site

Only for viewing the schema - no mapping possible.

Takeaway

If you map on a level besides the SSA/Tenant level, make sure the content you are mapping for resides at the level you map, and that queries executed also run in the same context. This means for a REST query you would set the API url to target the full URL of a site collection if that’s where the mapping is – https://server/site/mysite/_api/search/query, and not https://server/_api/search/query.

Read more at https://support.office.com/en-us/article/Manage-the-search-schema-in-SharePoint-Online-d4fab46d-ba41-4c03-9d4c-32b5b33198b6 and https://technet.microsoft.com/en-us/library/jj219667.aspx

Wednesday, February 10, 2016

Scenario where you do not want to use output caching in ASP.NET (MVP/WebAPI)

Always fun to get to the bottom of weird issues when you didn’t write the code yourself.

Take a look at the following method declaration.

[OutputCache(Duration = 3600, VaryByParam = "none", Location = OutputCacheLocation.Any, NoStore = true)]
public async Task<JsonResult> GetCurrentUser()
{
    var userId = UserHelperFunctions.GetCurrentUserId();
    var userName = UserHelperFunctions.GetUserName(userId);
    var hasPrivilege = <check admin access>

This is what happens, User A logs in and retreives user data, and within the next hour, every other visitor will get user A’s information. Also pay attention to line 6

Wow!

There are ways to solve this, for example like in this SO question, but for now I’ll set it to the below to make sure it’s not cached at all.

[OutputCache(NoStore = true, Duration = 0, VaryByParam = "*")]

A small alfanumeric sorter with numeric padding

I just got a support task where I’m pulling data from an API and need to sort the items based on a chapter like key. The data looks like this:

  • ST.1.1
  • ST.1.2
  • ST.1.10
  • ST.2.20a

The data is coming back random, and if I do a pure alfanumeric sorter, then ST.1.10 will sort above ST.1.2, which is lexically incorrect. Since I know I will never have numbers above three digits (ST.999.999), I figured I could sort it all by padding the numbers. This is the solution I ended up with, and if you have a smarter one let me know :)

I’m matching right to left in the regular expression as this makes it easier to replace the string as indexes are not changed when inserting characters.

class Program
{
    static void Main()
    {
        var demo = new List<string> { "ST.1.1", "ST.1.10", "ST.1.3c", "ST.1.3a", "ST.1.2" };
        demo = demo.OrderBy(PadName).ToList();
        demo.ForEach(Console.WriteLine);
        //output is: ST.1.1 ST.1.2 ST.1.3a ST.1.3c ST.1.10
    }

    static readonly Regex _reNumber =
       new Regex(@"\d+", RegexOptions.Compiled | RegexOptions.RightToLeft);
    static string PadName(string name)
    {
        foreach (Match match in _reNumber.Matches(name))
        {
            string newNumber = match.Value.PadLeft(3, '0');
            name = name.Remove(match.Index, match.Length)
                       .Insert(match.Index, newNumber);
        }
        return name;
    }
}

Tuesday, February 2, 2016

"I Want to Believe" - Is Microsoft abandoning the enterprise features of enterprise search?

image
I want to believe that Microsoft has not abandoned the enterprise in enterprise search, and that the good people behind the FAST seearch engine still are allowed to deliver well tested and proven features around the SharePoint search space.

If features are removed, as they were from FAST ESP to FAST Search 2010 from SharePoint, and from FAST Search 2010 for SharePoint to SharePoint 2013, then remove them first after you have tried to advocate the use of them. Just because a feature is not used by the average SharePoint consultant/pro-user, does not mean it’s not valuable. It might be that they just don’t know about it, or don’t know how to put it to good use.

To set the scene for this article, enterprise search is the practice of making content from multiple enterprise-type sources, such as databases and intranets, searchable to a defined audience. And over the years, companies have learned to expect a certain set of features from solutions delivering enterprise search. Wikipedia lists enterprise search having five components:
  • Content awareness
  • Content processing and analytics
  • Indexing
  • Query processing
  • Matching

The present

Before Microsoft dropped FAST ESP and Search Server as separate products they were a player in Gartner’s magic quadrant for Enterprise Search. But as they no longer have stand-alone search offerings, you won’t find them there today. When Microsoft was a part of the quadrant they were in the upper right corner for being both visionary and having the capability to execute. If Microsoft had a separate product in the quadrant today, would they still be in that coveted position?

On the execution part I think yes, as SharePoint 2013 is widely rolled out and SharePoint Online tenants are popping up at a steady pace. Visionary wise, I’m not so sure. With SharePoint 2013 Microsoft gave companies the query rule framework. If you know how to utilize it, the query rule framework is one of the coolest search features out there when developing search-based solutions. When it comes to the indexing and query features coming from FAST ESP, however, we have seen a decline in the feature set. Where FAST ESP gave you full control and FAST Search 2010 for SharePoint locked it down in terms of full customization, SharePoint 2013 has ridden itself of much of the capabilities to tune relevance and control how indexing works. In retrospect, Microsoft seem to have cut some corners on the SharePoint 2013/SPO search features and on the testing of the features that are included.

And don’t be fooled to think Microsoft is not aware of this. There are people out there who care about search functionality, like me, who have in both official and unofficial channels voiced their opinion on the state of Microsoft search within SharePoint.

On-premises features

Don’t get me wrong that it’s all dark ages, because there are some very good features in SharePoint search today that indeed are enterprise grade. I already mentioned the query rule framework, and dictionary extraction and content enrichment via a web service are also prime examples. The enrichment capability makes it a lot easier to do massaging of content to accomplish simple contextual and targeted experiences for the end-user. Could they have been even better? Sure. Those two features are better than FAST Search 2010 for SharePoint offered, but inferior to what FAST ESP offered.

For those not familiar with enriching capabilities, you use them to look at the text and metadata of the content being indexed and then add on even more content to the search index. That could be categorizations or classifications, adding synonyms to disambiguate terms, or applying any business logic you deem necessary.

On the query side, you can still create custom rank profiles used when querying the content to improve the order of how items are returned for users. It’s not for the faint of heart, but once you know how, it’s a very powerful feature – enterprise grade for sure. I would still hold FAST ESP and FAST Search 2010 for SharePoint as winners over the current rank profile offerings, mainly because machine learning plays a part today. Machine learning is hard to comprehend and to tune manually. In theory, machine learning should be superior, if the learning is done correctly.

To sum it up, SharePoint 2013 on-premises and the upcoming SharePoint 2016, which has the exact same feature set as 2013, hits all the five categories for enterprise search mentioned in Wikipedia, but not as well as FAST ESP and FAST Search 2010 for SharePoint did.

Enter hybrid crawl and SharePoint Online

If you decide to take the plunge for the recently hyped hybrid crawl where you index on-premises data into the SharePoint Online search index, or move to SharePoint Online all together, then the story effectively changes. You can no longer do enrichment of the indexed data, and the options for custom rank models are very limited – or useless in my opinion. This might sound harsh, but from a technical perspective, it is what it is.

Effectively point #2 (content processing) and point #5 (matching) of enterprise search have been set back some 10-15 years. Using third-party connector frameworks can mitigate this, but then complexity and cost increase.

Read my blog post Why Hybrid Crawl in SharePoint is a cold hot potato for more thoughts around hybrid crawl.

The future

Where Lucene and Elastic Search-based solutions speak to the developers and allows full flexibility in all parts of the search lifecycle, Microsoft has transformed its FAST search engine from being a flexible kit car to a Prius. I suspect the gasoline engine is still there at its core, but the ease of going electric seems very appealing and friendly to most. Had they gone all-in for a ludicrous mode like a Tesla it would have been a different story.

As of now, if you look to hybrid crawl or SharePoint Online, you do get a cheaper scalable solution, but at the cost of stepping back a decade in functionality. Effectively removing the “enterprise” in enterprise search. Just because you can index a lot of data and have enterprise customers, doesn’t mean it’s enterprise grade search.

Is Office Graph the new messiah?

The Office Graph is definitively a fresh breath into the enterprise search space and the only new search feature launched for SharePoint since SharePoint 2013 entered the market on October 11th, 2012 (and I’m not considering hybrid crawl a search feature per say). To explain in crude terms what the Office Graph does for users, it brings in a new way to rank content based on what you and people like you work with and work on based on recent usage and interaction with content.

Indeed, very useful, although I’m not sure today’s Delve application is the killer user experience to utilize the full potential of the Office Graph. I’ve played with the Office Graph myself, as has others, but I haven’t yet been smart enough to create something useful that I cannot live without on a day-to-day basis.

My Collabograph application, built using the Office Graph, is a fun gizmo playing into the Delve analytics space, but I don’t really need it (and the API has also been discontinued).

That said, Gartner might easily have moved Microsoft up in the visionary quadrant for launching the Office Graph feature, but is it good enough to keep them there in the long run?

Summary

Search capabilities in all flavors of SharePoint are still good, and certainly a good contender for enterprise search. Where other solutions lack the wide range of quality, third-party connectors adhering to the source systems strict security control on content access that SharePoint has, SharePoint truly shines. But on the technical side of what you can do with the content during indexing and querying, open source-based solutions and the players in the Gartner magic quadrant looks to have the upper hand today. My wish is that Microsoft should fill in the gaps they forgot back in October of 2012, and also build upon this even further. More capabilities will in general yield more innovation and excellent showcase solutions.

Perhaps the second coming, or comeback if you will, from Microsoft for enterprise grade search is when v2 of the Office Graph programmatic interfaces are made public and we can push all kinds of content and signals into the Office Graph. Or have Microsoft put down their gloves and left the enterprise search arena for good, moving all focus to battle Google and Amazon? That is a large-scale battle with more money to be made for sure.

I’m still a believer in Microsoft as an enterprise search player, and although I have doubts at the current time about the current feature set improving in the near future, I quote Fox Mulder from X-files saying: “I want to believe”.