How to: Use the SharePoint 2013 Content Enrichment Web Service

Posted Tuesday, April 8, 2014 3:47 PM by CoreyRoth

The Content Enrichment Web Service (CEWS) allows you to extend the functionality of SharePoint 2013 Search.  Using CEWS, a developer can send the values of managed properties to an external web service and return new or modified managed properties to include in the index.  The process involved implementing a custom WCF service and then registering it with PowerShell.  The PowerShell cmdlet specifies which properties go into and out of the service.

This post has been cross-posted to MSDN Code where you can download a working sample and deploy it.

This example will take the values of the Author and LastModifiedTime managed properties and write a new string such as "Modified by <author> on <LastModifiedTime>." to the managed property TestProperty.  This property need to be created first prior to trying to use your Content Enrichment Web Service.  The property should be configured as type Text with the following attributes: Query, Search,Retrieve, and Refine.

CEWSNewManagedProperty

To get started, create a new WCF Service Project called ContentEnrichmentExampleService.

CEWSVisualStudioNewService

Once the project is created, you can delete the default service Service1.svc and IService.cs as it won't be needed.

Next, you will need to add a reference to the following assembly. 

  • microsoft.office.server.search.contentprocessingenrichment.dll

This assembly can be found in the folder Installation Path\Microsoft Office Servers\15.0\Search\Applications\External.

Now, we need to create the service to do the content enrichment processing.  Create a new service called ContentEnrichmentExampleService.svc. 

CEWSVisualStudioNewService2

Delete the file IContentEnrichmentExampleService.cs as it will not be needed.  The custom service instead inherits from IContentProcessingEnrichmentService.

Now we can start adding our code to ContentEnrichmentProcessingExampleService.svc.cs.  This code will retrieve the values from the input properties, create our new output property TestProperty and send it back to the search index.

Start by adding using statements to the assembly we added.

using Microsoft.Office.Server.Search.ContentProcessingEnrichment;

using Microsoft.Office.Server.Search.ContentProcessingEnrichment.PropertyTypes;

The interface that the class is inheriting from will be shown as broken since you deleted it.  Change it instead to inherit from IContentProcessingEnrichmentService.

public class ContentEnrichmentExampleService : IContentProcessingEnrichmentService

Add a ProcessedItem collection to hold the output managed property values from the service.

private readonly ProcessedItem processedItemHolder = new ProcessedItem

{

    ItemProperties = new List<AbstractProperty>()

};

Then, Implement the ProcessItem method.  This method receives the input managed properties and allows you to write code to generate the output managed properties.

public ProcessedItem ProcessItem(Item item)

{

 

}

Inside the ProcessItem method, initialize the ErrorCode and ItemProperties.

processedItemHolder.ErrorCode = 0;

processedItemHolder.ItemProperties.Clear();

We then, need to Create a new output managed property named TestProperty.  The property object takes types based on what type of managed property you defined.

var testProperty = new Property<string>();

testProperty.Name = "TestProperty";

Now we are going to retrieve the managed properties using a simple lamdba expression.  Remember that the names of properties are case sensitive and need to match exactly how it shows on the Search Schema page.  You also need to cast the object to the appropriate type.  Since the Author managed property is a multi-valued property, we need to use List<string>.  The LastModifiedTime is a date so we use a DateTime type.

var authorProperty = item.ItemProperties.FirstOrDefault(i => i.Name == "Author") as Property<List<string>>;

var writeProperty = item.ItemProperties.FirstOrDefault(i => i.Name == "LastModifiedTime") as Property<DateTime>;

Now, we need to verify that the properties aren't null.

if ((authorProperty != null) && (writeProperty != null))

{

 

}

We are then going to write out a new string to TestProperty in the format Modified by {Author} on {LastModifiedTime}.  Since Author supports multiple values, only the first value was used.  This value goes in the Value property.  Once we set the value, we have to add it processedItemHolder so that it can send the values back to the search index.

testProperty.Value = string.Format("Modified by {0} on {1}.", authorProperty.Value.First(), writeProperty.Value);

processedItemHolder.ItemProperties.Add(testProperty);

Return the processItemHolder

return processedItemHolder;

At this point, we can run and debug our service using F5.  Leave the service running as it will be called when doing a full crawl.

To register the service with SharePoint we use using the New-SPEnterpriseSearchContentEnrichmentConfiguration cmdlet.  Use the following PowerShell script  to register the Content Enrichment Web Service.  Verify that the Endpoint parameter contains the correct URL to your service.  The example below has the location used in the source code I provided.  If you start from scratch or you have deployed you service to a remote server, then you will need to update the address.

$ssa = Get-SPEnterpriseSearchServiceApplication  $config = New-SPEnterpriseSearchContentEnrichmentConfiguration  $config.Endpoint = "http://localhost:54641/ContentEnrichmentExampleService.svc"  $config.InputProperties = "Author", "LastModifiedTime"  $config.OutputProperties = "TestProperty"  $config.SendRawData = $false  $config.Timeout = 30000  $config  Set-SPEnterpriseSearchContentEnrichmentConfiguration –SearchApplication $ssa –ContentEnrichmentConfiguration $config

The InputProperties parameter specifies the managed properties sent to the service.  The OutputProperties specifies the managed properties returned by the service.  Note, that both are case sensitive.  All managed properties referenced need to be created in advance.  Set the Timeout propety higher to give yourself sufficient time to debug.  For a complete reference on parameters, see this MSDN reference.

After registering your content enrichment service, start a full crawl.  Again, ensure that your Content Enrichment Web Service is running in the debugger.  While it is crawling, you can set breakpoints as desired. 

To verify the functionality after the crawl is complete, issue a query using REST in the browser like the one below.

http://server/_api/search/query?querytext='*'&selectproperties='title,path,author,testproperty'

This query will return every item in the index and include the new TestProperty field.  You can verify that the new property was included and has the expected result as shown in the example below.

CEWSRESTAPIQuery

I hope this gets you started with Content Enrichment Web Services.  I have a few follow-up posts to include on some more of the PowerShell parameters, but I hope this helps.

Again, you can find the complete source code and PowerShell script on MSDN Code.  Feel free to leave me a comment if you run into an issue or have a question.

Comments

# How to: Use the SharePoint 2013 Content Enrichm...

Wednesday, April 9, 2014 8:09 AM by How to: Use the SharePoint 2013 Content Enrichm...

Pingback from  How to: Use the SharePoint 2013 Content Enrichm...

# How to: Use the SharePoint 2013 Content Enrichm...

Wednesday, April 9, 2014 8:48 AM by How to: Use the SharePoint 2013 Content Enrichm...

Pingback from  How to: Use the SharePoint 2013 Content Enrichm...

# SharePoint 2013: Recopilatorio de enlaces interesantes (XXXII)! | Pasi??n por la tecnolog??a...

Pingback from  SharePoint 2013: Recopilatorio de enlaces interesantes (XXXII)! | Pasi??n por la tecnolog??a...

# SharePoint 2013: Recopilatorio de enlaces interesantes (XXXII)!

Thursday, May 1, 2014 1:51 PM by Blog de Juan Carlos González en Geeks.MS

Como cada nuevo mes, os dejo un nuevo recopilatorio de enlaces y recursos sobre plataformas SharePoint

# re: How to: Use the SharePoint 2013 Content Enrichment Web Service

Thursday, September 25, 2014 4:40 AM by Mark Slavik

Hi Corey, this is really interesting stuff. Is there any way to catch items that have been deleted via the 2013 pipeline.

Thanks...Mark

# re: How to: Use the SharePoint 2013 Content Enrichment Web Service

Tuesday, January 13, 2015 6:42 PM by CoreyRoth

@Mark unfortunately, I don't think anything is there to allow you to do that.

# re: How to: Use the SharePoint 2013 Content Enrichment Web Service

Monday, March 16, 2015 9:42 AM by Björn

Hi Corey, I also wrote you an E-Mail, but here again my question.

Which license is necessary to run these services on an OnPrem Installation. Do we need enterprise cals for that? If yes, how much. Do we need an Enterprise cal for each user that will use the new generated managed properties?

# re: How to: Use the SharePoint 2013 Content Enrichment Web Service

Tuesday, March 17, 2015 12:06 PM by CoreyRoth

Hi Bjorn.  I don't think I ever saw an e-mail from you.  However to answer your question, this requires Enterprise Edition of SharePoint Server but it does not require an ECAL for each user.  This is because the feature isn't really associated with an individual user.  If you were to license just a few ECAL users like the ones for the accounts that create the properties that should be sufficient.  As always, double check with Licensing to be sure because I could be wrong, but that is my understanding.

# Microsoft Ignite 2015–Day 4

As we near the end of Ignite, day 4 was still filled with some great technical sessions. I’ll give you a rundown of everything from SQL High Availability, best practices for branding and page design in SharePoint Online to scaling SharePoint 2013 search

# re: How to: Use the SharePoint 2013 Content Enrichment Web Service

Wednesday, November 25, 2015 10:01 AM by Sean

Hi Corey, great article can you give some idea on how to add multiple managed properties and not just the one (I would be looking at adding 6 separate items as a minimum)?

# re: How to: Use the SharePoint 2013 Content Enrichment Web Service

Tuesday, September 20, 2016 6:23 AM by Riddhi

hey in your ignite video you also showed the same for office 365 can you share powershell on how you did that?

AFAIK, you mentioned using search REST API to read the item, to update the properties and update the source item back with the enriched data?

Leave a Comment

(required)
(required)
(optional)
(required)