December 28, 2016

Search cannot see some metadata tagged documents SharePoint

Greetings to fellow SharePoint developers and admins.. here is one more issue about crawling managed metadata terms and how to solve it.

The objective was to have a custom document search by terms. In SharePoint 2013, we have a document library with couple of managed metadata fields (some are single while others are multi value). Based on some logic, a custom code solution will assign (tag) term values to documents. Tagged documents are ultimately to be found using a custom search page by passing 2 term names and their Guids. Example:


The search page contains Content Search Web Parts. Each one contains a custom query that uses query strings to retrieve all items tagged with multiple values of either or both of the 2 terms used. Example:

{|owstaxIdSites:{QueryString.Sites}*} {|owstaxIdSiteCategory:{QueryString.SiteCategory}*}

The Issue:
The CSWP query show some documents, but do NOT show other documents even if they were tagged with exactly the same terms.

Here is the list of symptoms:
- Document item is tagged with valid terms (we can see/edit the terms with no issue in SharePoint UI in DispForm/EditForm views respectively on all fields)
- Document is crawled successfully and can be viewed by default search page when search by title
- No issue from crawl log regarding said documents themselves nor their metadata terms
- If I just just edit the document and Save, then run crawl it might fix the issue for some documents!
- Also, if I call SystemUpdate() on item from code (without making any other changes), then crawl it will be fixed as well!
- I noticed that for affected documents, all terms from all metadata fields are not showing under under any query.

Failed attempts: 
Tried the following methods,but Non of the below worked:
- Tried re-run Incremental Crawl.
- Tried partial re-index target site or content source (to force search to consider documents as 'dirty' as if the property 'vti-searchversion' was updated) then run Incremental-Crawl.
- Tried Full crawl.

Hmm, so it's not about the search crawl, not the document, not the terms, but something with how the terms are stored in metadata columns. But from the UI, everything seems normal..

Root Cause:

So I paid another visit to the custom code part. After careful checking, I found a bug in code where there are cases where it inserted multiple copies of the same term in one of the fields.

Example: from browser the metadata column show one term 'POLICIES', but from code it has multiple duplicates:


Apparently, when multiple copies of the same term are pushed by code to a managed metadata column, then it's it is a column validation issue. The SharePoint UI will not complain but the search crawler somehow will have a problem with it and will drop the managed metadata crawling for the all managed columns of that item altogether. The document item itself is with other non-managed columns will be crawled.

On the other hand, I can conclude the default SharePoint metadata controls offer some validation/fix. When editing the document from UI, and just click Save without making any changes, the duplicate terms might get removed.

Check you code to see where it tries to update managed metadata fields with duplicate copies of the same term. No exception will be thrown so you have to carefully check your logic.

FYI, This is a related post where merging terms in store lead to the same issue


November 29, 2016

How To Show Solution Name For Each Feature In SharePoint

The objective:
Ever occurred to you that you loose track of what solutions relates to what features enabled at site or site collection level? Yeah I've been in that situation too.
The objective is to add new field to Site features & Site Collection features page (/_layouts/ManageFeatures.aspx) showing a link of Solution name for each feature.

First, let's have a look for on urls of different pages:
Site features:         /_layouts/ManageFeatures.aspx
Site Coll. features: /_layouts/ManageFeatures.aspx?Scope=Site
WebApp features: /_admin/ManageWebAppFeatures.aspx?WebApplicationId={Id}&IsDlg=1
 (from CA -> select WebApp -> Manage Features)
Farm features:       /_admin/ManageFarmFeatures.aspx
 (from CA -> System Settings -> Manage farm features)

In this post, I am going to customize 'ManageFeatures.aspx' page only. However, the same procedure below can be applied to other pages. (Note: influenced by code from following post , I decided that I should start customizing my solution

OOB Page Details:
Let's first understand contents of ManageFeatures.aspx page and how it works:

- Fields: FeatureIcon(image), FeatureTitleDescription(Text), FeatureActiveButton(button), FeatureActiveState(text)

- Page calls a user control (~/_controltemplates/FeatureActivator.ascx) to render the table of features
<asp:Content ContentPlaceHolderId="PlaceHolderMain" runat="server">
 <wssuc:FeatureActivator runat="server" ID="featact" />
 <SharePoint:FormDigest runat=server />

- Page also calls another control ( ~/_controltemplates/FeatureActivatorItem.ascx) to do the feature logic
   DataItem='<%# Container.DataItem %>'

The feature logic is at code files:
- Microsoft.SharePoint.WebControls.FeatureActivator.cs
- Microsoft.SharePoint.WebControls.FeatureActivatorItem.cs

Customization Steps for SP2010 & SP2013:

1. First, make sure you backup the original manageFeatures.aspx page & user controls files (FeatureActivator.ascx & FeatureActivatorItem.ascx).

2. Make a copy for the following: page & user controls, then rename all with postfix with "Ex":
 ManageFeaturesEx.aspx, FeatureActivatorEx.ascx, FeatureActivatorItemEx.ascx

3. Do the following changes to edit 3 files:

- In ManageFeaturesEx.aspx file:
 Set the Src parameter to refer to FeatureActivatorEx.ascx
 <%@ Register TagPrefix="wssuc" TagName="FeatureActivator" src="~/_controltemplates/FeatureActivatorEx.ascx" %>

- In FeatureActivator.ascx file:

    1. Set the Src parameter to refer to FeatureActivatoritemEx.ascx
    <%@ Register TagPrefix="wssuc" TagName="FeatureActivatorItem" src="~/_controltemplates/FeatureActivatorItemEx.ascx"     %>

    2. Add new column header ("Solution") under <colgroup> add: <col width="10%" />

    3. Add new column content:
    *** For SP2010 ***
    Under <tr> add:       
      <th scope="col" id="FeatureSolution" class="ms-vh2" style="padding-bottom: 4px"><SharePoint:EncodedLiteral             runat="server" text="From Solution" EncodeMethod='HtmlEncode'/></th>

    *** For SP2013 ***
    Under <tr> add:       
      <th scope="col" id="FeatureSolution" class="ms-vh2-nofilter" style="padding-bottom: 4px"><SharePoint:EncodedLiteral         runat="server" text="From Solution" EncodeMethod='HtmlEncode'/></th>

- In FeatureActivatorItem.ascx file:
    1. Only for SP2013: First Add these 2 lines to import SharePoint namespaces
    <%@ Import Namespace="Microsoft.SharePoint" %>
    <%@ Import Namespace="Microsoft.SharePoint.Administration" %>

    2. insert the following code in page after the header tags & before first <td> tag:
    <script runat = "server">
        SPSite Site = SPContext.Current.Site;
        SPSolutionCollection FarmSolutions = SPContext.Current.Site.WebApplication.Farm.Solutions;
        private string getSolutionNameFromFeatureDef(SPFeatureDefinition featDef)
        string result = string.Empty;
        Guid FeatureSolutionID = featDef.SolutionId;
        if (FeatureSolutionID != Guid.Empty)
            SPSolution solution = FarmSolutions[FeatureSolutionID];
            if (solution != null)
                result = solution.SolutionFile.DisplayName;
        return (result);

        public string getSolutionName()
        string result = string.Empty;
            Guid FeatureID = new Guid(this.FeatureId);
            SPFeatureDefinition CurrentFeatureDefinition;
            SPFeature CurrentFeature = Site.Features[FeatureID];
            if (CurrentFeature != null)
                // feature is activated at site collection
                CurrentFeatureDefinition = CurrentFeature.Definition;
                // feature is activated at Farm or other level
                CurrentFeatureDefinition = SPFarm.Local.FeatureDefinitions[FeatureID];
            result = getSolutionNameFromFeatureDef(CurrentFeatureDefinition);
        catch (System.Exception)
            // do nothing when exception
        return (result);

    3. Then add a new <td> cell at the end of file:
      <td class="<% Response.Write(this.CssClass); %>" style="padding-top: 4px; padding-bottom: 4px; ">
            <table width="100%" cellpadding="0" cellspacing="0" border="0">
            <tr><td class="ms-vb2"><% Response.Write(getSolutionName());%></td></tr>

    Note: for SP2013 make sure to postfix any '_layouts/' with '15' to target the new hive

4. Copy the 3 files to hive (14 for SP2010 or 15 for SP2013) in all related WFE servers:
 - Copy controls FeatureActivatorEx.ascx, FeatureActivatorItemEx.ascx to \TEMPLATE\CONTROLTEMPLATES folder
- Copy page ManageFeaturesEx.aspx to \TEMPLATE\LAYOUTS folder

That's it! no need for IIS reset. Now you can enjoy having the solution info across all sites collection & site features across the farm.

Note: the solutions column will be empty when no wsp found for target feature (example: for OOB MS features that were stapled in with site definition). Also, solutions column will not support user solutions (sandboxed solutions).

September 19, 2016

Quick notes on debugging code workflows in SharePoint

You have a custom workflow in SharePoint 2010 or 2013, where it reads some configuration keys from web.config then continue to do some business logic.

What's the issue?
So you added the 2 keys to your web application web.config on your WFE server. The workflow worked just fine a couple of time, and was able to read the config keys, then the next time it failed to read one or both of the keys! The occurrence is sporadic and its kind of hit or miss!
2 cases from ULS: first time it fails to find key1, then next time it fails to find key2!

Let's debug:
First things first! Workflows run by both w3wp & owstimer processes.
In SharePoint, workflow always start running by the World Wide Web Publishing service (w3wp.exe). If it goes to sleep (by a pause or delay activities or excessive load), then the 'SPTimerV4' SharePoint Foundation Workflow Timer service (owstimer.exe) will wake it up and run it.
SharePoint will automatically decide which Workflow Timer service will run the workflow. The operation can occur on any server running the "Microsoft SharePoint Foundation Workflow Timer Service" service.

To debug at run time, attach VS 2010 to the w3wp and owstimer process to detect which one it is.

After deploying new code the timer service must be restarted (iisreset will not suffice)
net stop sptimerv4
net start sptimerv4

To avoid wasting time figuring out why it didn't read the config keys, let's use a method to tell us the file name and machine name the code was trying to read the config key. Use it like this:

string KeyValue = getConfigKeyValue("MyKey");

Here is the nice method:

private string getConfigKeyValue(string keyName)
            string KeyVal = string.Empty;
            bool error = false;
                KeyVal = ConfigurationManager.AppSettings[keyName];
                if (KeyVal == null)
                    error = true;
            catch (Exception)
                error = true;

            if (error)
                KeyVal = string.Format("The key '{0}' was not found in file {1} in machine {2}",
                    keyName, AppDomain.CurrentDomain.SetupInformation.ConfigurationFile, Environment.MachineName);
                throw new Exception(KeyVal);
            return KeyVal;

That way we can find and set the missing key easily.

Example 1:
System.Exception: The key 'key1' was not found in file D:\inetpub\WWWROOT\wss\VirtualDirectories\7001\web.config in machine DevServerWFE1

Example 2:
System.Exception: The key 'key1' was not found in file C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\BIN\OWSTIMER.EXE.Config in machine DevServerWFE1