Property Promotion and Demotion in SharePoint

Once I experienced a peculiar  behaviour in the search functionality in one of the SharePoint environments. The task was to simply move a bunch of office documents in a SharePoint 2010 farm to a SharePoint 2013 farm. Yes, it was a document migration. The document metadata was scheduled to be updated in the new document library based on data in a CSV file. Had to do some digging and troubleshoot the issue.

The documents downloaded from SP 2010 web site had the metadata embedded within the document properties. The SP 2013 site had the "parserenabled" property set to true.

This has caused the metadata embedded within the documents to be Promoted to the site. That resulted in causing inconsistencies in the Search results.

So whats Property promotion and demotion? Property promotion refers to the process of extracting values from properties of a document and writing those values to corresponding columns on the list or document library where the document is stored. Property demotion is the same process in reverse.

So in my case, what happened was, the documents properties had been automatically promoted to SharePoint. When the document was added to the document library, the Document Parser extracts document property values and writes to a Property Bag (to an instance of the IParserPropertyBag Interface). And then SharePoint reads the properties from the Property Bag and promotes the applicable property values to the library based on Document Content Type or the Library Schema.

When the "parserenabled"  property is set to true in a site, SharePoint automatically captures the properties set within the office document and Promotes them.

Properties are only promoted or demoted if they match the list's columns that apply to the document.

If you need to disable document parser and stop this behaviour you could do that with a simple PowerShell script

$site = new-object microsoft.sharepoint.spsite("http://mySharePointSite.com/SubSite")
$web = $site.openweb()
$web.parserenabled = $true
$web.update()

If you don't need the Promoted properties to be available in the Search results, you could do that as well. Go to Search Service Application via the SharePoint Central Administration. Go to "Search Schema". Select the Managed Property where the document's Parsed property is set as a Mapped Crawled property. Remove the mapping with the Parsed Property.

CRUD operations with the SharePoint lists using REST and AngularJS

In the below examples I have used the 'ABC Mappings' as my SharePoint list. The CRUD operations are done against the items in this list.

Create

var payload = '';
var itemCreated = '';

        payload = { SourceField: item.SourceField, SheetName: item.SheetName, CellCoordinate: item.CellCoordinate, DataType: item.DataType, __metadata: { type: 'SP.Data.ABCMappingsListItem' } };

        var siteUrl = window.location.protocol + "//" + window.location.host + _spPageContextInfo.webServerRelativeUrl;

        $http({
            method: 'POST',
            url: siteUrl + "/_api/web/lists/getByTitle('ABC Mappings')/items",
            data: payload,
            async: false,
            headers: {
                "Accept": "application/json;odata=verbose",
                "Content-Type": "application/json;odata=verbose",
                "X-RequestDigest": $("#__REQUESTDIGEST").val()
            }
        }).success(function (data, status, headers, config) {

            itemCreated = data.d.ID;

        }).error(function (data, status, headers, config) {

            alert('error');

        });

Read

var siteUrl = window.location.protocol + "//" + window.location.host + _spPageContextInfo.webServerRelativeUrl;

        $http({
            method: 'GET',
            url: siteUrl + "/_api/web/lists/getByTitle('ABC Mappings')/items?$select=ID,SourceField,SheetName,CellCoordinate,DataType",
            headers: { "Accept": "application/json;odata=verbose" }
        }).success(function (data, status, headers, config) {
            $scope.items = data.d.results;
        }).error(function (data, status, headers, config) {
            alert('error');
        });

Update

Initially you need to find the list item type. For this we can use the below REST call

http://<weburl>/_api/lists/getbytitle('Document Library Name')?$select=ListItemEntityTypeFullName

The function call:

        //Variables
        var payload = '';
        var itemUpdated = '';
        var itemId = item.ID;

        //Create item
        payload = { SourceField: item.SourceField, SheetName: item.SheetName, CellCoordinate: item.CellCoordinate, DataType: item.DataType, __metadata: { type: 'SP.Data.ABCMappingsListItem' } };

        var siteUrl = window.location.protocol + "//" + window.location.host + _spPageContextInfo.webServerRelativeUrl;

        $http({
            method: 'POST',
            url: siteUrl + "/_api/web/lists/getByTitle('ABC Mappings')/items(" + itemId + ")",
            data: payload,
            async: false,
            headers: {
                "Accept": "application/json;odata=verbose",
                "Content-Type": "application/json;odata=verbose",
                "IF-MATCH": "*",
                "X-HTTP-Method": "MERGE",
                "X-RequestDigest": $("#__REQUESTDIGEST").val()
            }
        }).success(function (data, status, headers, config) {


        }).error(function (data, status, headers, config) {

            alert('error');

        });


Delete

 var siteUrl = window.location.protocol + "//" + window.location.host + _spPageContextInfo.webServerRelativeUrl;

        $http({
            method: 'POST',
            url: siteUrl + "/_api/web/lists/getByTitle('ABC Mappings')/items(" + itemId + ")",
            async: false,
            headers: {
                "Accept": "application/json;odata=verbose",
                "Content-Type": "application/json;odata=verbose",
                "IF-MATCH": "*",
                "X-HTTP-Method": "DELETE",
                "X-RequestDigest": $("#__REQUESTDIGEST").val()
            }
        }).success(function (data, status, headers, config) {


        }).error(function (data, status, headers, config) {

            alert('error deleting');

        });

"sorry this site hasn't been shared with you" Error on site home page

If the site was working perfectly earlier and suddenly you get the error : "sorry this site hasn't been shared with you" on your site's home page.

But you notice that you still can access the site content via "_layouts/15/viewlsts.aspx".

You might also notice, that you could browse certain SP libraries but not "Site Pages" or "Site Assests" libraries.

If this is the case:

Go to the Central Admin and restart the "Microsoft SharePoint Foundation Web Application" service.

Go back to work. :D

Using Fiddler to test a POST call to a custom WCF REST service hosted in SharePoint 2013

Under Parse tab change the call type to "POST"
Content type should be set to json
In the "Request Body" specify the parameters to be sent as shown below


Fiddler Image:

To create a custom WCF REST service hosted in SharePoint 2013, follow this link

Sending POST call to WCF REST web method - SharePoint

When it comes to SharePoint usage there are many examples in the web about REST GET calls, but not so much on POST. So here goes..

My target REST method signature is as follows

public List<Details> GetDetails(string ids)

The method declaration in the Interface is as below:

[OperationContract]
[WebInvoke(Method = "POST", UriTemplate = "GetDetails", BodyStyle = WebMessageBodyStyle.Wrapped,
 RequestFormat = WebMessageFormat.Json,
 ResponseFormat = WebMessageFormat.Json)]
List<Details> GetDetails(string ids);

The trick here is making the web message body style "Wrapped"

Now from the client-side you make a call to the method.
In my example I use the service invocation within a AngularJS function. You send the parameter information in the data

    $scope.TestMethod = function () {
        var endpointAddress = "http://myTestSite.com/sites/TestSite/_vti_bin/CustomService.svc";
        var callurl = endpointAddress + "/GetDetails";
        $.ajax({
                type: "POST",
                url: callurl,
                contentType: "application/json; charset=utf-8",
                dataType: "json",
                data: '{"ids": "18"}',
                success: function () {
                    alert('success');
                },
                error: function () {
    alert('error');
}
            });

    };

Update Managed Property & Manually Re-Index a SharePoint Library

The SharePoint crawler picks up content that has changed since the last crawl and updates the search index.

When you change a managed property or when you change the mapping of crawled and managed properties then the site must be re-crawled before he changes are reflected in the search index. Since changes are made in the search schema and not to the actual site, the crawler will not automatically re-index. To make sure that the changes are crawled and fully re-indexed, you must request a re-indexing. The site content will be re-crawled and re-indexed so that you can start using the managed properties in queries, query rules and display templates.

In my case I updated several managed property. After the update a re-indexing of the SharePoint library was required for the items to show up in the search.

Mapping the Managed property with the Crawled property

• In the Central Administration go to the Search Service Application.
• Go to Search Schema
• Search for you Managed Property

• Click on property in the search results.
• Make the property Queryable so that this appear in search items
• Click on the "Add a mapping" button and map with the crawled property

[Note: you can do the mapping other way around too.. By searching the crawled property and map the managed property to the crawled property, depending on your scenario]

• Save changes

Re-Indexing the SharePoint library

• Go to the  library settings page of the target SP library.
• Select Advanced settings
• Scroll down and click on the "Reindex Document Library" button
• Click on "Reindex document library" button as shown below

• The re-indexing will happen in the next scheduled crawl.

Hide Elements Associated to Master pages in Dialogs

Once I had a minor UI issue with a Modal dialog. Yet I thought to blog about it as it may be useful. An image attached to the master page and a button in the page overlapped in a Modal popup. The issue was due to the fact that the image's z-index was higher than the button according to the build up of the DOM tree. Once of the options is to send the image back using the z-index. As a quick fix can adjust the styling of the div containing the image as:

.myImageDiv {

    z-index: -1;

}

Anyway decided to remove the image only for popups. How can this be done? Well, in SharePoint 2013 you can use the UI class "ms-dialogHidden" in your div's. So when the page is displayed in a Modal popup, the divs containing this CSS class are not displayed.

<div class="myImageDiv ms-dialogHidden"></div>

That's it ! 

Best practices when querying SharePoint lists/libraries

1) Golden rule - Never iterate through the whole list to filter/check-on couple of items in the list. List operations are heavy. Filter the required items needed maybe through a CAML query.

2) Use the SPQuery class if your objective is to get items from a single list

3) In your CAML query set ViewFields to filter the fields which you need. In the same way you can use SPList.GetItemByIdSelectedFields() method to get selected fields. Setting ViewFields makes the query more efficient

4) Use the SPSiteDataQuery class to get results from multiple lists. Here you can specify the web and lists which you are targeting at.

Example:

SPSiteDataQuery myQuery = new SPSiteDataQuery();
myQuery.query = "Your CAML query goes here";
myQuery.Webs = @"<Webs Scope=""SiteCollection"" />"; //Returns data from all webs in the current site collection
//myQuery.Webs = @"<Webs Scope=""Recursive"" />"; // will return from current web and its child webs

myQuery.Lists = @"<Lists ServerTemplate=""101"" />"; //101 refers to document libraries
myQuery.ViewFields =  @"
                                            <FieldRef Name=""Field One"" />
                                            <FieldRef Name=""Field Two"" />
                                           ";

Access User Profile Service using Powershell

function GetInfo([string]$userAccount)
{

 $mailAttribute = "WorkEmail"

 $mail = ""

 $serviceContext = Get-SPServiceContext -Site $siteUrl
 $profileManager =  New-Object  Microsoft.Office.Server.UserProfiles.UserProfileManager($serviceContext);

if($profileManager.UserExists($userAccount))
{
        $userProfile = $profileManager.GetUserProfile($userAccount)
        $mail = $userProfile[$mailAttribute].Value
}

return $mail
}

What happens when Enable-SPSessionStateService -DefaultProvision

Powershell command

Enable-SPSessionStateService -DefaultProvision 

The changes to the SQL server and web site are as follows:

1) SQL server
In the sql server a new Database will be created as 

"SessionStateService_706b1a65e15248618a0090cd50ed1823"

2) Web.config changesIn the web.config file he below entry is inserted

<system.web>
...
<sessionState mode="SQLServer" timeout="60" allowCustomSqlDatabase="true" sqlConnectionString="Data Source=shaamils_SqlServer;Initial Catalog=SessionStateService_706b1a75e15248618a0090cd60ed4813;Integrated Security=True;Enlist=False;Pooling=True;Min Pool Size=0;Max Pool Size=100;Connect Timeout=15" />
</system.web>

<modules runAllManagedModulesForAllRequests="true">
...
<add name="Session" type="System.Web.SessionState.SessionStateModule" />
</modules>

Open Modal dialog in SharePoint

You can use a JavaScript function for this purpose.

1) When you don't require a return from the Modal dialog.

    function openModalWindow(strPageURL) {
        var options = {
            url: strPageURL,
            title: "Title goes here",
            showClose: true,
            width: 600,
            height: 400,
            allowMaximize: false
        };

        SP.SOD.execute('sp.ui.dialog.js', 'SP.UI.ModalDialog.showModalDialog', options);
    }

2) When a return value is required to be captured by the Parent page

Include a callback parameter in the options specified

Option 1: Include in the options parameter collection

function openModalWindow(strPageURL) {
        var options = {
            url: strPageURL,
            title: "Title goes here",
            showClose: true,
            width: 600,
            height: 400,
            allowMaximize: false
            dialogReturnValueCallback: CloseDialogCallback
        };

        SP.SOD.execute('sp.ui.dialog.js', 'SP.UI.ModalDialog.showModalDialog', options);
    }

Option 2: Place the code after other option parameters are defined

var options = {
                url: strPageURL,
                allowMaximize: false
                showClose: true
            };
            options.dialogReturnValueCallback = Function.createDelegate(null, CloseDialogCallback);

            SP.SOD.execute('sp.ui.dialog.js', 'SP.UI.ModalDialog.showModalDialog', options);


In the invoked function, you could specify the actions to do once the Modal Dialog closes, such as refreshing the parent page

function CloseDialogCallback(dialogResult, returnValue) {
            if (returnValue.d) {
               /*Do something */
            }
            if (dialogResult == SP.UI.DialogResult.OK) {
                SP.SOD.execute('sp.ui.dialog.js', 'SP.UI.ModalDialog.RefreshPage', SP.UI.DialogResult.OK);
            }
        }

3) Closing the Modal Dialog (Add script to the page in the Modal Dialog)

This can be done in two ways

• Without referencing the SP.UI.Dialog.js

function closeOnCancel() {
            window.frameElement.commonModalDialogClose(0 /* 0 for cancel */, 'Cancelled');
        }

function closeOnOk() {
            window.frameElement.commonModalDialogClose(1 /* 1 for ok */, 'OK result');
        }

- OR - 

• using the SP.UI.Dialog.js

Reference the JavaScript on your page.
<script src="/_layouts/15/SP.UI.Dialog.js" type="text/javascript"></script>

function closeOnCancel() {
            SP.UI.ModalDialog.commonModalDialogClose(SP.UI.DialogResult.Cancel, 'Cancelled');
        }
function closeOnOk() {
            SP.UI.ModalDialog.commonModalDialogClose(SP.UI.DialogResult.OK, 'OK result');
        }

To call these JavaScript functions from server side code use ScriptManager:

ScriptManager.RegisterStartupScript(this, this.GetType(), "closeScript", "closeOnOk();", true);

However, the final JavaScript function might give out a JavaScript error in some Internet Explorer versions.

AJAX call from Visual Web Part - SharePoint

1)

In the JavaScript function create the JSON object. Here the stringify method is used to construct the object which contains the parameters to be sent. The function in the JavaScript method of the aspx page is as below:

    function CallMyPage(itemValue, paramTwo, paramThree, paramFour) {

        var final = { myItem: itemValue, parameterTwo: paramTwo, parameterThree: paramThree, siteUrl: paramFour };

        $.ajax({

            type: "POST",

            url: paramFour + "/_layouts/15/MyFeatureFolder/WebMethods.aspx/MyWebMethod",

            data: JSON.stringify(final),

            contentType: "application/json; charset=utf-8",

            dataType: "json",

            success: OnSuccess,

           failure: function (response) {

             alert(response.d);

          }

       });

    }

2)

In the code behind within a SPGridView I have a HyperLink control. I set the JavaScript method call in the code behind as:

editLink.NavigateUrl = "javascript:CallMyPage('" + itemGuid.Text + "','" + MyParamTwo + "' ,'" + MyParamThree + "', '" + SPContext.Current.Web.Url + "');"; 

3)

In my approach instead of a web service I am using an application page (WebMethods.aspx) to get the job done. In the application page servicing the AJAX call, I create the web method

        [System.Web.Services.WebMethod]

        public static string MyWebMethod(string myItem, string parameterTwo, string parameterThree, string siteUrl)

        {

            string result = string.Empty;

            try 

            {

                //TODO - Process request and return result

            }

            catch (Exception)

            {

                //Exception Handling

            }

            return result;

4)

On success

    function OnSuccess(response) {

        if (response.d) {           

            /* Handle  result */            

        }

    }

Note: I have used the jquery-1.11.2.min.js file in my solution

Adding Custom Properties to a Visual Web Part

When the Visual web part is generated in Visual Studio (Eg; VS 2013). It contains mainly 3 components. The web part file, user control a ASCX file and the code behind for the user control

The web part is generated with the following code in the CreateChildControls() method

private const string _ascxPath = @"~/_CONTROLTEMPLATES/15/....../MyWebPartUserControl.ascx";

protected override void CreateChildControls()
{
  Control control = Page.LoadControl(_ascxPath);
  Controls.Add(control);
}

Here, the .ascx could not access the custom web part properties.
When adding custom properties to the Visual web part,  few changes are required.

1) Create a property in the user control, of the web part class type.

public MyVisualWebPart ParentWebPart { get; set; }

2) Set the web part to the user controls parent property

Inside the web part file's CreateChildControls() method change the code to:

MyVisualWebPartUserControl control = (MyVisualWebPartUserControl)Page.LoadControl(_ascxPath);
//Set the web part to the users controls parent property                                                                             
control.ParentWebPart = this;
Controls.Add(control);    

Now the custom web part properties are accessible from the user control !                                                                                                              

3) Add the required custom properties into the web part file

        private string customPropertyOne;                           
        [Category("My WebParts Custom Settings"),           
        Personalizable(PersonalizationScope.Shared),          
        WebBrowsable(true),                                              
        WebDisplayName("Custom Property One"),            
        WebDescription("This is my first custom property")] 
        public string CustomPropertyOne                             
        {                                                                             
            get { return customPropertyOne; }                      
            set { customPropertyOne = value; }                    

        }                                                                            

4) Access the property in the ASCX file

Now you can access the custom property via:

ParentWebPart.CustomPropertyOne

Services available in SharePoint Foundation 2013

SharePoint Foundation only has a few services available when compared to the Server version.
Namely they are:

• App Management Service
• Business Data Connectivity Service
• Lotus Notes Connector
• Search Service Application
• Secure Store Service
• State Service
• Usage and Health data collection

the name SPSecurity does not exist in the current context

In SharePoint when using SPSecurity you might come across this error "the name SPSecurity does not exist in the current context"


Probably you are using a sandbox solution. You cannot use SPSecurity in sandbox solutions. Instead, go with a Farm solution in order to run this code.

Basic PowerShell for SharePoint and the Equivalent STSADM used in Earlier Version

This is a simple list down of basic PowerShell commands used in command line deployments in SharePoint. [SP 2010 and 2013]  &
The equivalent STSADM commands used in SharePoint 2007 ... back in the day ;)


1. Adding a new WSP to farm solution

SP 2010/2013 PowerShell	Add-SPSolution -LiteralPath "C:\FolderName\Solution.wsp"
SP 2007  Stsadm	stsadm -o addsolution -filename "Solution.wsp" [Assuming command runs from the correct folder location]

2. Deploy WSP 

SP 2010/2013 PowerShell

Install-SPSolution -Identity "Solution.wsp" -WebApplication "http://WebSiteUrl" -CASPolicies -GACDeployment -Local -Force AllWebApplications vs WebApplication - All allows the wsp to be deployed in all SP web applocations. WebApplication - allows the wsp to be deployed to a target web app CASPolicies - Allows Code Access Security Policies to be deployed [optional] GACDeployment - Allows DLL installation in the global assembly cache [optional] Force [optional] Local - Deploys only in the current server [optional]

SP 2007  Stsadm

stsadm -o deploysolution -name "SolutionName" -url "Site URL" -immediate -allowgacdeployment -allowcaspolicies -force instead of -url, you can use -allcontenturls  instead of -immediate, you can use -time "time to deploy" GACDeployment - Allows DLL installation in the global assembly cache [optional]

3. Activate feature

SP 2010/2013 PowerShell	Enable-SPFeature –identity "FeatureName" -URL http://WebSiteUrl
SP 2007  Stsadm	stsadm -o activatefeature -id feature_ID -url http://WebSiteUrl -force [Instead of -Id, you can use -filename or -name with appropriate values]

4. De-activate feature

SP 2010/2013 PowerShell	Disable-SPFeature –identity "FeatureName" -URL http://WebSiteUrl
SP 2007  Stsadm	stsadm -o deactivatefeature -id feature_ID -url http://WebSiteUrl [Instead of -Id, you can use -filename or -name with appropriate values]

5. Retract WSP 

SP 2010/2013 PowerShell	Uninstall-SPSolution -Identity "Solution.wsp" -WebApplication http://WebSiteUrl
SP 2007  Stsadm	stsadm -o retractsolution -name "Solution.wsp" -url http://WebSiteUrl -immediate  [Instead of -immediate, you can use -time "TimeToRun"]

6. Remove WSP from SharePoint farm

SP 2010/2013 PowerShell	Remove-SPSolution -Identity "Solution.wsp"
SP 2007  Stsadm	stsadm -o deletesolution -name "Solution.wsp"

This version of Visual Studio does not have the following project types installed or does not support them

You might come across instances where you get the below message when opening SharePoint solutions in Visual Studio 2012.

"This version of Visual Studio does not have the following project types installed or does not support them"

1) The immediate remedy that pops into mind is to update the Visual Studio 2012 version.

The latest update for Visual Studio 2012, as of now is Update 4 (Released: November 13, 2013). Could get it from the link:  Visual Studio 2012 Update

2) Once that's done, you might get the following message on project load.

"An error occurred while trying to load some required components. Please ensure that prerequisite components are installed"

• Microsoft Web Developer Tools
• Microsoft Exchange Web Services

Possibilities are that either

  You don't have the "Microsoft Office Developer Tools for Visual Studio 2012" OR

  The installed tool is not complete.

Solution:

• If the tool is already installed, uninstall through Control panel,
• Download  Web Platform Installer   
• Use the Web Platform Installer to install the tool.

SharePoint Web Parts Maintenance Page

SharePoint web part maintenance pages allows to manage (reset, delete) the web parts embedded into a particular page. It's useful in situations where its required to remove any web parts from a page which has caused a page crash.

For an instance if the page urls is:

http://localhost:1000/SitePages/Home.aspx

you can get the maintenance page by typing in "?contents=1" at the end of the URL string. Below is a sample:

http://localhost:1000/SitePages/Home.aspx?contents=1

This works the same in  SharePoint 2007, 2010 and 2013

SharePoint Debug - Attach to Worker Process

Each SharePoint web application is run under a dedicated worker processes. Specially when it comes to DEV environments, when debugging a solution it's best to attach the code only to the target worker process. This avoids the hassle among the DEV team :) . It's pretty simple  to find the worker process related to the target web application. Follow these simple steps.


1) Start Command prompt with Administrator privileges
2) Navigate to C:\Windows\System32\inetsrv directory
3) Type in appcmd list wp


You get the list of  worker processes along with the web application which is running on it. Now in the Visual Studio IDE go to debugger and attach your code by selecting the appropriate worker process. Under the set of w3wp exe's you can pinpoint the exact one.

Get-SPWebApplication is not recognised as the name of the cmdlet

When you try to execute SharePoint specific commands through PowerShell, might come across a message

"Get-SPWebApplication is not recognized as the name of the cmdlet ......"

The SP specific commands are not being recognized in PowerShell at this point. Adding a snap in would do the trick. Type the below command in your PowerShell  window.

Add-PSSnapin Microsoft.Sharepoint.Powershell

That's it. Provided that the rights in DB are there, you should be able to execute SharePoint based commands now.