Corey Roth [MVP] : SPC09

Get to know the Refinement Web Part in SharePoint 2010 Enterprise Search

CoreyRoth — Mon, 15 Mar 2010 19:22:57 GMT

In MOSS 2007, people often extending the search results experience by using the Faceted Search Web Parts. People really liked these so it looks like Microsoft decided to implement their own version called the RefinementWebPart (also known as the Refinement Panel). If you are familiar with the Faceted Search Web Parts at all, you will notice there are a lot of similarities in the way things are implemented. If you’re not familiar with what I’m talking about, it’s this web part that allows users to drill down into a set of search results based upon managed properties and other criteria. Let’s take a look at a quick example.

On the left there, you will see a set of refinements that we get out of the box. I didn’t have to do anything to configure these at all. There are a number of refinements built in including file type, site, author, modified date, and taxonomy. You can also easily create your own based on managed property which we will see here shortly.

Let’s take a look at some of the options on this web part. If we edit the page and then edit the web part, we will be able to see the options. If you’re on a small screen (or a small window in the case of a VM), you will have to scroll right to see the web part properties.

Since this is a search web part, you will see the familiar Cross-Web Part query ID. This should be synced up to whatever else you are using on the page. Usually it is set to User query. The next section is where we can configure the refinement. Make note of the Use Default Configuration checkbox. If you don’t uncheck this, anything you customize will not be saved. Each thing that you can filter on is called a category by the RefinementWebPart. The Filter Category Definition property is an XML field where each category is specified. We’ll also look at that later in the post. Other properties to note here. The Accuracy Index is the number of results it looks at to determine things to refine. I assume this it mainly there to keep things performing well. What this does mean is that if there is something unique to refine on but it doesn’t occur until result number 51, then it will not be included. You can also configure how many categories to display. This is set to 6 by default. The last thing I will point out is that you can configure how the web part displays its information using XSLT (not shown in the screenshot above).

Now let’s look at how we can add our own managed property to the category list. You should of course confirm the managed property works and that you can query on it first (do a full crawl if necessary). We then, just need to take a look at the XML for the Filter Category Definition.

<?xml version="1.0" encoding="utf-8"?>
<FilterCategories>
  <Category    Title="Result Type"    Description="The file extension of the item"    Type="Microsoft.Office.Server.Search.WebControls.ManagedPropertyFilterGenerator"    MetadataThreshold="5"    NumberOfFiltersToDisplay="4"    MaxNumberOfFilters="0"    SortBy="Frequency"    SortDirection="Descending"    SortByForMoreFilters="Name"    SortDirectionForMoreFilters="Ascending"    ShowMoreLink="True"    MappedProperty="FileExtension"    MoreLinkText="show more"    LessLinkText="show fewer">
    <CustomFilters MappingType="ValueMapping" DataType="String" ValueReference="Absolute" ShowAllInMore="False">
      <CustomFilter CustomValue="Adobe PDF">
        <OriginalValue>pdf</OriginalValue>
      </CustomFilter>
      <CustomFilter CustomValue="Email">
        <OriginalValue>eml</OriginalValue>
        <OriginalValue>msg</OriginalValue>
        <OriginalValue>exch</OriginalValue>
      </CustomFilter>
      <CustomFilter CustomValue="Excel">
        <OriginalValue>odc</OriginalValue>
        <OriginalValue>ods</OriginalValue>
        <OriginalValue>xls</OriginalValue>
        <OriginalValue>xlsb</OriginalValue>
        <OriginalValue>xlsm</OriginalValue>
        <OriginalValue>xlsx</OriginalValue>
      </CustomFilter>
      <CustomFilter CustomValue="Image">
        <OriginalValue>tif</OriginalValue>
        <OriginalValue>tiff</OriginalValue>
      </CustomFilter>
      <CustomFilter CustomValue="Lotus Notes">
        <OriginalValue>nsf</OriginalValue>
      </CustomFilter>
      <CustomFilter CustomValue="One Note">
        <OriginalValue>one</OriginalValue>
      </CustomFilter>
      <CustomFilter CustomValue="PowerPoint">
        <OriginalValue>odp</OriginalValue>
        <OriginalValue>ppt</OriginalValue>
        <OriginalValue>pptm</OriginalValue>
        <OriginalValue>pptx</OriginalValue>
      </CustomFilter>
      <CustomFilter CustomValue="Publisher">
        <OriginalValue>pub</OriginalValue>
      </CustomFilter>
      <CustomFilter CustomValue="Text">
        <OriginalValue>odt</OriginalValue>
        <OriginalValue>txt</OriginalValue>
        <OriginalValue>url</OriginalValue>
        <OriginalValue>csv</OriginalValue>
      </CustomFilter>
      <CustomFilter CustomValue="Visio">
        <OriginalValue>vdw</OriginalValue>
        <OriginalValue>vdx</OriginalValue>
        <OriginalValue>vsd</OriginalValue>
        <OriginalValue>vss</OriginalValue>
        <OriginalValue>vst</OriginalValue>
        <OriginalValue>vsx</OriginalValue>
        <OriginalValue>vtx</OriginalValue>
      </CustomFilter>
      <CustomFilter CustomValue="Webpage">
        <OriginalValue>ascx</OriginalValue>
        <OriginalValue>asp</OriginalValue>
        <OriginalValue>aspx</OriginalValue>
        <OriginalValue>htm</OriginalValue>
        <OriginalValue>html</OriginalValue>
        <OriginalValue>jhtml</OriginalValue>
        <OriginalValue>js</OriginalValue>
        <OriginalValue>mht</OriginalValue>
        <OriginalValue>mhtml</OriginalValue>
        <OriginalValue>mspx</OriginalValue>
        <OriginalValue>php</OriginalValue>
        <OriginalValue></OriginalValue>
      </CustomFilter>
      <CustomFilter CustomValue="XML">
        <OriginalValue>xml</OriginalValue>
      </CustomFilter>
      <CustomFilter CustomValue="Word">
        <OriginalValue>doc</OriginalValue>
        <OriginalValue>docm</OriginalValue>
        <OriginalValue>docx</OriginalValue>
        <OriginalValue>dot</OriginalValue>
        <OriginalValue>nws</OriginalValue>
      </CustomFilter>
      <CustomFilter CustomValue="Zip">
        <OriginalValue>zip</OriginalValue>
      </CustomFilter>
    </CustomFilters>
  </Category>
  <Category    Title="Site"    Description="Which site this document is from"    Type="Microsoft.Office.Server.Search.WebControls.ManagedPropertyFilterGenerator"    MetadataThreshold="5"    NumberOfFiltersToDisplay="4"    MaxNumberOfFilters="20"    SortBy="Frequency"    SortByForMoreFilters="Name"    SortDirection="Descending"    SortDirectionForMoreFilters="Ascending"    ShowMoreLink="True"    MappedProperty="SiteName"    MoreLinkText="show more"    LessLinkText="show fewer" />
  <Category    Title="Author"    Description="Use this filter to restrict results authored by a specific author"    Type="Microsoft.Office.Server.Search.WebControls.ManagedPropertyFilterGenerator"    MetadataThreshold="5"    NumberOfFiltersToDisplay="4"    MaxNumberOfFilters="20"    SortBy="Frequency"    SortByForMoreFilters="Name"    SortDirection="Descending"    SortDirectionForMoreFilters="Ascending"    ShowMoreLink="True"    MappedProperty="Author"    MoreLinkText="show more"    LessLinkText="show fewer"    />
  <Category    Title="Modified Date"    Description="When the item was last updated"    Type="Microsoft.Office.Server.Search.WebControls.ManagedPropertyFilterGenerator"    MetadataThreshold="5"    NumberOfFiltersToDisplay="6"    MaxNumberOfFilters="0"    SortBy="Custom"    ShowMoreLink="True"    MappedProperty="Write"    MoreLinkText="show more"    LessLinkText="show fewer" >
    <CustomFilters MappingType="RangeMapping" DataType="Date" ValueReference="Relative" ShowAllInMore="False">
      <CustomFilter CustomValue="Past 24 Hours">
        <OriginalValue>-1..</OriginalValue>
      </CustomFilter>
      <CustomFilter CustomValue="Past Week">
        <OriginalValue>-7..</OriginalValue>
      </CustomFilter>
      <CustomFilter CustomValue="Past Month">
        <OriginalValue>-30..</OriginalValue>
      </CustomFilter>
      <CustomFilter CustomValue="Past Six Months">
        <OriginalValue>-183..</OriginalValue>
      </CustomFilter>
      <CustomFilter CustomValue="Past Year">
        <OriginalValue>-365..</OriginalValue>
      </CustomFilter>
      <CustomFilter CustomValue="Earlier">
        <OriginalValue>..-365</OriginalValue>
      </CustomFilter>
    </CustomFilters>
  </Category>
  <Category    Title="Managed Metadata Columns"    Description="Managed metadata of the documents"    Type="Microsoft.Office.Server.Search.WebControls.TaxonomyFilterGenerator"    MetadataThreshold="3"    NumberOfFiltersToDisplay="3"    MaxNumberOfFilters="20"    ShowMoreLink="True"    MappedProperty="ows_MetadataFacetInfo"    MoreLinkText="show more"    LessLinkText="show fewer" />
  <Category    Title="Tags"    Description="All managed metadata of the documents and social tags"    Type="Microsoft.Office.Server.Search.WebControls.TaxonomyFilterGenerator"    MetadataThreshold="3"    NumberOfFiltersToDisplay="3"    MaxNumberOfFilters="20"    ShowMoreLink="True"    MappedProperty="ows_MetadataFacetInfo,popularsocialtags"    MoreLinkText="show more"    LessLinkText="show fewer" />
</FilterCategories>

I went ahead and posted the entire XML because it’s worth seeing. For the most part the schema is pretty easy to follow. The Category element defines each thing to refine and then it has some basic configuration items such as the number of filters to display. You specify the name of the managed property to use in the MappedProperty attribute. The MetadataThreshold property is the number of results that have to be returned with that property in order to do refinement. If you are familiar with the schema used by Faceted Search, you will see a similar concept with the CustomFilter elements. These allow you map a value into something more readable. For example instead of display xlsx, it displays Excel.

One more thing I will point out is that the Category element has a Type attribute. So far I have seen ManagedPropertyFilterGenerator and TaxonomyFilterGenerator. These both inherit from RefinementFilterGenerator. None of its sealed surprisingly which means you could actually write your own custom filter for the refinement web part. That’s pretty cool. I’m not sure why I would need to yet, but you never know what you might want to be able to customize.

I want to add my own managed property, so I just add a category to the end like this.

<Category    Title="Color"    Description="Use this filter to restrict results by color" Type="Microsoft.Office.Server.Search.WebControls.ManagedPropertyFilterGenerator"    MetadataThreshold="5"    NumberOfFiltersToDisplay="4"    MaxNumberOfFilters="20"    SortBy="Frequency"    SortByForMoreFilters="Name"    SortDirection="Descending"    SortDirectionForMoreFilters="Ascending"    ShowMoreLink="True"    MappedProperty="Color"    MoreLinkText="show more"    LessLinkText="show fewer"    />

All I did was copy the category that was used for author and it works great. You can tweak the individual settings if you like. Here is what my custom managed property looks like.

How does this all work though. Well it’s pretty simple. Just copy any link that the web part displays and we’ll see that it makes use of the new r query string property. It will be URL encoded but you can easily decode it and see the magic. In this case if I want to view Red products, the query string has a r parameter of the following.

r=color="Red"

If I wanted to view files modified in the last 24 hours, it uses the new >= operator.

r=write>="3/14/2010”

Don’t you just love that you can easily query against dates now using the keyword query syntax? I think the r keyword is very interesting and I think it opens the door for some very interesting customizations of the search results page in the future.

Intro to SharePoint 2010 Development: How to Build and Deploy a Web Part

CoreyRoth — Mon, 15 Feb 2010 20:17:41 GMT

If you are already familiar with SharePoint 2010, you already know how easy it is to build and deploy a web part now. However, this post is for those that don’t keep up with SharePoint as some of us do and may not realize how the development experience has improved so much. My post How to Build and Deploy a Web Part is by far the most popular post on DotNetMafia.com. I wanted to make today’s post just as a point to show you how much less work is involved in deploying a web part. I am going to group this post into sections in a similar manner as I did the post for the WSS3 post.

Environment

There can be entire talks about what the best way to develop is now, but we’ll start with the simplest. Although you can install SharePoint on Windows 7 and directly develop on it, most people are going to say stick with a virtual machine and run Windows Server 2008 R2. It’s certainly simpler to get all of the prerequisites installed if you stick with Windows Server. The benefits to developing directly on a machine with SharePoint on it are so great now that I would recommend against remote debugging (although you still can). The SharePoint Root (or the 12 hive as you called it) is now the 14 hive and is located at the predictable path below.

C:\Program Files\Common Files\microsoft shared\Web Server Extensions\14

Coding the Web Part

Here is where things start to change. Instead of creating a class library and adding references to the SharePoint DLLs, we simply use one of the new included SharePoint project templates as you can see here.

Start by using the Empty SharePoint Project template. Also make sure you have it set to .NET Framework 3.5 as SharePoint does not run under .NET Framework 4.0 (don’t get me started). You’ll notice you have many different project templates to choose from. Most of these can also be used once you create an empty project. On the next dialogue, pick farm solution. I’ll go into the difference between sandboxed and farm solutions, but more than likely you are going to use farm solutions every time. You also need to specify the URL to your server. You can change that if you want but the default value will probably work for you in this case.

This gives us a solution that looks like this.

Now we are ready to build our new web part. If you bring up the add new item context menu, you will see a number of choices for the types of new SharePoint Project Items (SPIs) that you can create. We’re going to choose Web Part in this case.

What is the Visual Web Part you ask? That’s just a user control which relates directly to my second most popular post on How to Deploy a User Control. Now we’re finally ready to add some code. We’re just going to take our code from the WSS3 post and use it here.

using System;
using System.ComponentModel;
using System.Runtime.InteropServices;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.WebControls.WebParts;
using Microsoft.SharePoint;
using Microsoft.SharePoint.WebControls;
 
namespace SharePointProject1.TestWebPart
{
    [ToolboxItemAttribute(false)]
    public class TestWebPart : WebPart
    {
        public TestWebPart()
        {
        }
 
        protected override void CreateChildControls()
        {
            base.CreateChildControls();
            Controls.Add(new Label() { Text = "My Test SharePoint 2010 Web Part (Hello World)!" });
        }
 
        protected override void RenderContents(HtmlTextWriter writer)
        {
            base.RenderContents(writer);
        }
    }
}

The only line of code I added here was the line to add the label and set its text. Everything else came from the template.

Describing the Web Part

In my WSS3 post, this is where I talked about building a .webpart file. Well, you don’t need to worry about that any more as Visual Studio creates it for you. Here is what solution explorer looks like after you add your first web part.

As you can see the .webpart file is already there as well as an elements.xml file for a feature to deploy the web part. The WSS3 post went on to talk about all of the things you need to know about building a feature. This is still good stuff to know, but its already taken care of for you. If you want to edit the basic feature information, just open it up in solution explorer and you get a nice new interface that looks like this.

I’m not going to waste space showing you the insides of the files it creates for you. Just know it creates them for you and it saves you a ton of time.

Deploying via Solution Package

In my WSS3 post, I explained how to create a cab.ddf and manifest.xml file. Well guest what? That is taken care of for you now as well. The Package.package file in the solution explorer provides another nice editor which allows you to choose with files go into the package. You don’t have to keep track of a thing any more, it just builds the package and takes care of it for you.

At this point, Visual Studio has created the .webpart file, the feature, and the solution package. However, we still need to deploy it and if we could debug it that would be even cooler right? Take a look at our new options in the Build menu.

We can build and rebuild just like any other project, but notice the options for Deploy, Package, and Retract. Those are all SharePoint functions. In this case, I want to deploy my solution. Choosing deploy, we see the following in the output window.

------ Build started: Project: SharePointProject1, Configuration: Debug Any CPU ------
SharePointProject1 -> C:\Code\SharePointProject1\bin\Debug\SharePointProject1.dll
Successfully created package at: C:\Code\SharePointProject1\bin\Debug\SharePointProject1.wsp
------ Deploy started: Project: SharePointProject1, Configuration: Debug Any CPU ------
Active Deployment Configuration: Default
Run Pre-Deployment Command:
Skipping deployment step because a pre-deployment command is not specified.
Recycle IIS Application Pool:
Recycling IIS application pool 'SharePoint - 80'...
Retract Solution:
Skipping package retraction because no matching package on the server was found.
Add Solution:
Adding solution 'SharePointProject1.wsp'...
Deploying solution 'SharePointProject1.wsp'...
Activate Features:
Activating feature 'Feature1' ...
Run Post-Deployment Command:
Skipping deployment step because a post-deployment command is not specified.
========== Build: 1 succeeded or up-to-date, 0 failed, 0 skipped ==========
========== Deploy: 1 succeeded, 0 failed, 0 skipped ==========

From inspecting the text of the log, you can see that Visual Studio compiled, created a package, reset my Application Pool, Added the Solution, Deployed the Solution, and activated the feature. Let’s check SharePoint and see if it’s really there.

Checking the web part gallery, we see our .webpart file. Let’s add it to a page and see how it looks. Edit any page and use add a web part and you will see the new interface for choosing a web part. It puts it in the Custom group by default.

One you hit and add finish editing, we see the web part working correctly.

You have to admit this is quite a bit easier than deploying a web part in SharePoint 3. What if you want to debug though? No problem. Just set a breakpoint and choose debug from the build menu like you would any other type of project.

As I mentioned earlier, if you are familiar with SharePoint 2010, this is nothing new to you. However, my point today is for those who shied away from SharePoint in the past because the development experience was far from optimal. Try it for yourself and you will see how easy it is to get up and running with your code. Even with pictures this post is half the size of the WSS3 post. That’s because it really is just that easy. I really think Visual Studio 2010 will open the way for a new round of SharePoint developers. Try it out today.

Enterprise Search Web Parts are still Sealed! Unseal them! (SP2010)

CoreyRoth — Wed, 09 Dec 2009 21:44:54 GMT

I am pretty passionate about Enterprise Search and with MOSS 2007 I saw customers really struggle because wildcard search was not offered out of the box and most of the web parts in Microsoft.Office.Server.Search.WebParts are sealed. Luckily, the most important one CoreResultsWebPart was in fact not sealed and I was able to hack together a wildcard search solution by inheriting it and using some hacky reflection. Since SPC09, we’ve been hearing “the search web parts are no longer sealed”, this unfortunately is not true at all. What they mean to say is that they have added a couple of override to CoreResultsWebPart that allow you to affect the query going in and the results going out. This is a nice win, but there is still more desired.

It took me asking a lot of people at SPC09 to finally get a straight answer. It was in the session, Customizing Search in SharePoint: Building Great Sites with Search, where I asked Harneet Sidhana at the microphone to get the story on why most web parts are still sealed. The answer? The remaining web parts are currently still sealed by design. The audience “booed”. Seriously, they did. I then asked to have the web parts unsealed and the audience applauded.

Let’s take a look at the web parts between MOSS 2007 and SP2010 Beta 2 and see what is sealed.

Web Part	MOSS 2007	SP2010 Beta 2
AdvancedSearchBox	Sealed	Sealed
CoreResultsWebPart	Unsealed	Unsealed
FederatedResultsWebPart	Unsealed	Unsealed
HighConfidenceWebPart	Sealed	Sealed
PeopleCoreResultsWebPart	Sealed	Sealed
PeopleRefinementWebPart		Unsealed
QuerySuggestionsWebPart		Unsealed
RefinementWebPart		Unsealed
SearchPagingWebPart	Sealed	Sealed
SearchStatsWebPart	Sealed	Sealed
SearchSummaryWebPart	Sealed	Sealed
TopFederatedResultsWebPart	Unsealed	Unsealed

Looking at this list, I don’t really see anything that has changed from sealed to unsealed. Do you? I will point out that CoreResultsWebPart has always been unsealed, but this is because PeopleCoreResultsWebPart inherits from it. Did you notice that anything built since the Infrastructure Update of MOSS 2007 has been unsealed?

Now I admit, the main reason we wanted to inherit from these controls was to do wildcard search. The Enterprise Search did a great job with the new keyword query improvements and this will be a non-issue now. However, can they really say we will never want to extend their web parts? Are they going to say that they have thought of every possible thing that the web part might possibly do and no one will ever extend it? As a programmer, I don’t think I would ever have the guts to make a claim like that. I know these web parts will require customization in the future. Give us the ability to do it.

I know you might be thinking, “We just wanted to inherit from these web parts because we wanted wildcard search. We have that now.” This is true and I am very thankful. This eliminates the need a lot, but more than likely we might need to make a change to something else though. Want to customize people search? What about best bets or the advanced search box? Most web parts in SharePoint are unsealed, so why are the search web parts sealed?

The PeopleCoreResultsWebPart still being sealed is a huge deal. For one, it doesn’t appear to support the new QueryManager class which means you can’t customize how it does queries or builds the results. I need to confirm this though. Take a look at the forums. They speak for themselves. Fifty results for the term wildcard in the Search forum. I am passionate about this topic because I have seen the needs of my customers. The fact that I might be stuck with another set of sealed web parts for another 3+ years is absolutely scary to me.

So what can we do? Please post a reply in the forum or leave a comment on the blog. We need to let them know our concerns while there is still time to make a change. Please, unseal the web parts. This post is cross posted to the SharePoint 2010 forums.

Getting by with bare minimum hardware in a SharePoint 2010 development environment (sp2010)

CoreyRoth — Mon, 07 Dec 2009 15:22:36 GMT

Let’s face it. A lot of companies would rather you be unproductive and waste hours a day rather than spend a couple of hundred bucks on some more memory. I don’t get it, but that’s the world so many of us are forced to live in. You have to beg and plead, write justification, get approval from a handful of VPs only to get some guy at the help desk to tell you that you don’t need any additional hardware. If that’s your case, I sympathize with you as I have been there plenty of times. When people starting seeing the hardware recommendations for SharePoint 2010, many were in shock. Most people were recommending 4 GB to 8 GB of memory just for the virtual machine itself. They are significantly higher than what you could get away with in MOSS 2007. After all with MOSS, I could get by with only 1 GB of memory allocated to a virtual machine. It wasn’t ideal, but it did work ok. If I had 2 GB of memory, things usually ran pretty smoothly. Maybe you will disagree, but that was the case for me at least. Keep in mind that we’re not talking about production here. We’re talking about a development virtual machine.

Today, I will focus primarily on memory since this is where the bottleneck is going to be. When it comes to developing, I have found that any modern CPU seems to do fine when it comes to performance. A faster hard disk is always nice, but really, it’s always lack of memory that causes issues. To date, I have set up three virtual machines on Beta 2: one with SharePoint Foundation, one with SharePoint Server 2010, and one with SharePoint Server 2010 with FAST Search installed. The memory requirements I have seen on these has varied vastly (as you would expect). For today’s post, I am considering a complete self-contained SharePoint 2010 environment on a virtual machine that includes Windows Server 2008 R2, Active Directory, and SQL Server 2008. Let’s start by looking at a summary of my findings.

Configuration	SharePoint Foundation	SharePoint Server 2010	SharePoint Server 2010 with FAST
Bare Minimum	2000 MB	3000 MB	4100 MB*
Visual Studio Running	2300 MB	3300 MB	4400 MB

Now, I’ll say these numbers are completely unofficial. You may have different results. If you have results you would like to share, please leave a comment. I quoted numbers for Visual Studio separately because you may not have Visual Studio running or you might be an admin or something. Obviously, the memory requirements of Visual Studio vary vastly at times depending on what you are doing, but this is just a guideline. These numbers will also vary depending on what services you have running in SharePoint. In this case, I have every service I can see to start running.

As you can see installing FAST adds considerable demand to your memory requirements. The reason I add the asterisk here is that I think it will probably consume more memory if I make it available.

So what you can take from this is if you just want a quick development environment and you don’t need the full SharePoint Server, you can use SharePoint Foundation to do some development work with relatively “low” memory requirements. You can further use even less memory by using an external SQL Server. On most of my installations, I have found SQL Server using 500 MB – 600 MB of memory. By moving that off to another server, you can really trim down your memory requirements. If you don’t need a domain controller on your server, you can shave a bit more off too (but don’t expect to gain a lot here).

Now, the last thing I will point out is this is what you can get by with. It’s not necessarily ideal. You may find that your hard drive is churning a lot as it is probably doing a lot of swapping. You may find yourself waiting a lot just for the SharePoint site to spin up or to compile and deploy something. It’s better than nothing though. Again, these are just some of the numbers I was seeing on my installations. If you have any numbers you would like to share, please leave a comment.

Turning on the Developer Dashboard in SharePoint 2010

CoreyRoth — Fri, 30 Oct 2009 18:42:01 GMT

The developer dashboard is a great new feature that developers can use to aid them in tuning performance on a page. This new functionality adds information to the bottom of any page in SharePoint that displays performance information and what SQL queries were executed to display the page. To my knowledge, there is currently no way in the UI to turn this on, so you can do this with a quick x64 console application. This is soon to become a quite popular code snippet I am sure.

SPPerformanceMonitor performanceMonitor = SPFarm.Local.PerformanceMonitor;
performanceMonitor.DeveloperDashboardLevel = SPPerformanceMonitoringLevel.On;

All you need is those two lines of code. When you are done with it, simply set the monitoring level to Off.

Will Virtualization die in SharePoint 2010 development?

CoreyRoth — Fri, 30 Oct 2009 17:45:00 GMT

This is already starting to be an interesting topic, so I wanted to post about it. We’ll start with the facts.

SharePoint 2010 only runs on 64 bit operating systems
In the past, SharePoint developers have pretty much always had to develop in a virtual environment
Windows Virtual PC does not support 64 bit guests but other non-Microsoft virtualization technologies do
SharePoint 2010 can run on Windows 7 / Vista x64
Most developers do not want to run a server OS on their development machine
Some developers may still be running 32 bit operating systems (ack!)

So what does this mean? For some reason, Microsoft has not given us desktop virtualization software that can run 64 bit guests (even though Hyper-V can). So I think this left Microsoft a choice. Get 64 bit guests added to Windows Virtual PC, recommend developers use a non-Microsoft virtualization technology, or get SharePoint working on Windows 7 and Vista. Now, adding 64 bit support I am sure is quite an undertaking and they definitely don’t want to recommend a competitor’s product. This means that making SharePoint work directly on the developer’s machine the obvious choice. After all it runs on top of IIS and IIS is pretty similar between Windows Server 2008, Windows 7, and Windows Vista (yes, I do know there are differences).

Now, I think it will be interesting to see what path developers choose here. Many developers I am sure probably already were using another virtualization software such as VMWare. For them, they are pretty much unaffected and will probably continue to developer in their VM. By now, I think most SharePoint developers have it engrained in them that they must develop on a VM, but maybe they will change their mind. People are so accustomed to thinking that development must be done in a VM, I hear them asking not when the beta 2 install will be available, but when will the VM be available? My answer is always well even if you have a VM, you aren’t going to be able to run it unless you are running Hyper-V or it was made with VMware.

Personally, I think I like the idea of running SharePoint on my laptop. I lose some flexibility that the VM gives me but I also gain some convenience of not having to maintain multiple versions of Visual Studio and what not. Will it affect performance of my laptop? Absolutely, but that just gives me a chance to ask for better hardware. :) Of course, if you are in a company where you have little control of your hardware, getting upgraded to Windows 7 x64 could be an issue.

UPDATE: @spmcdonough reminded me in a comment about booting a VHD natively with Windows 7. This option was actually brought up in the Patterns & Practices talk at SPC. They of course did not mention any competing virtualization solutions. :) It certainly is a way to get some good VHD performance by running the operating system natively. This may be a good option for some people but probably not for me. I don’t want to have to leave everything behind in my Client OS while I am doing my development. Sure you could install all the various things that you use from your client environment onto that VHD but then aren’t you just replacing your client OS with a server OS? It is an interesting option though. It may not work for me, but it might for some. Although I have to say I might consider it if I had a second machine available.

UPDATE (01/04/2010): So Beta 2 has been out for a while and I have actually tried a number of options. I started out trying VMWare Server running on my Windows 7 laptop. I was still looking for a free option. This worked ok, but not great. Since VMWare Server is designed for servers, it has no issues about taking every available resource on your machine for your VMs whether it needs it or not. This made going back to the host machine just to check Outlook or visit a web site somewhat painful. Adding more memory helped, but it still wasn’t ideal. So I decided to try Sun VirtualBox. The first thing I liked about it was the fact that I was able to take my VMWare hard disk file and boot it directly with VirtualBox without going through any type of conversion process. It just worked. I tried it with several VMs and it really does work great. The performance of the VM also feels a bit better and my host machine still has plenty of resources left to do whatever I need it to. If you are looking for a free option and don’t feel like dual booting, I definitely recommend giving VirtualBox a try.

So what do you think you are going to do? Are you going to install it locally on your Windows 7 machine? Go with VMware? Install Windows Server 2008 on your laptop maybe? Quit SharePoint development? :) There are pros and cons to each approach. I’d love to hear your plan. Leave a comment and tell us your plan.

What’s new in Services on Server in SP2010

CoreyRoth — Thu, 22 Oct 2009 20:00:00 GMT

Curious to know what the Services on Server page looks like now? Well, then this is the post for you. Instead of just having the 6 or so services that you had in MOSS 2007, there are many more services listed on the Services on Server page. Here is what your list might look like (pending changes of course).

Access Database Service
Application Registry Service
Business Data Connectivity
Central Administration
Document Conversion Launcher Service
Document Conversion Load Balancing Service
Excel Calculation Services
Lotus Notes Connector
Managed Metadata Web Service
Microsoft SharePoint Foundation Incoming E-mail
Microsoft SharePoint Foundation Subscription Settings Service
Microsoft SharePoint Foundation Web Application
Microsoft SharePoint Foundation User Code Service
PerformancePoint Service
PowerPoint Service
Search Query and Site Settings Service
Secure Store Service
SharePoint Foundation Search
SharePoint Server Search
User Profile Service
User Profile Synchronization Service
Visio Graphics Service
Web Analytics Data Processing Service
Web Analytics Web Service
Word Automation Services
Word Viewing Service

As you get more familiar with the concept of Service Applications, you will notice that many of the items on this list are service applications.

New Events in SharePoint 2010

CoreyRoth — Thu, 22 Oct 2009 15:52:56 GMT

Microsoft really answered the call of developers by adding several new events that developers had been wanting in MOSS 2007. Event better, Visual Studio 2010 really makes it easy by using the new Add Event Receiver wizard. It provides an interface with all of the possible things you might want to implement and then creates them in a class for you. When it comes to feature receivers, I’ll point out that SPFeatureReceiver no longer requires you to implement every method. You can now just create Event Handling methods for the events you care about.

Here is a list of the new events.

FeatureUpgrading
WebAdding
WebProvisioned
ListAdding
ListAdded
ListDeleting
ListDeleted
WorkflowStarting
WorkflowStarted
WorkflowPostponed
WorkflowCompleted

Hopefully, what the events do is obvious from their names. Of course, the name of these events might change at some point in the future. I would like to see the WebProvisioned event be renamed WebAdded personally.

Lots of Databases in SharePoint 2010

CoreyRoth — Wed, 21 Oct 2009 20:08:58 GMT

If you are like me when you get a new product, you look at every possible little thing you can find to see what’s new or different. That’s why I post this today because I figured you might be interested in what the databases might looks like in the next version. Of course, as usual this is subject to change in the future. In a typical installation, MOSS 2007 had a good number of databases. It looks like SharePoint 2010 more than doubles the number. My particular installation had 22 databases. I think this number can even grow even more as you activate more Service Applications. Most of the names are obvious, but I have commented on a few of them. Some of the databases have spaces in their names. The names are determined based upon how the service application was named.

Application_Registry_Service_<guid> – not sure if this is for the old Business Data Catalog or not.
Bdc_Service_db_<guid>
Managed_Metadata_Service_<guid> – likely for the new metadata term store
People_ProfileDb_<guid>
People_SocialDb_<guid> – new social tagging feature
People_SyncDb_<guid>
People_SyncDb_<guid>_Service
People_SyncDb_<guid>_Sync
PerformancePoint Monitoring Service_<guid>
Search_Service_Application_CrawlStoreDb_<guid> – labeled as admin in search dashboard
Search_Service_Application_Db_<guid> – labeled as index in search dashboard
Search_Service_Application_PropertyStoreDb_<guid> – labeled as query in search dashboard
Secure_Store_Service_Db_<guid>
SharePoint_AdminContent_<guid> – CA database
SharePoint_Config
SSO
StateService_<guid>
Web AnalyticsServiceApplication_Reporting_Db_<guid>
Web AnalyticsServiceApplication_Staging_Db_<guid>
Word Automatiation Services_<guid>
WSS_Content – content database
WSS_Logging – likely for analytics

As you can see you have more databases to backup now.

Use those version numbers in SharePoint 2010

CoreyRoth — Wed, 21 Oct 2009 17:59:45 GMT

With MOSS, it seemed like most web part developers tended to avoid changing the version of their DLL, because it lead to numerous headaches as you had to update the .webpart file. You also had to update any page that you might have the web part on as well so that it would load the new version. What’s nice about SharePoint 2010 is that your manifest file in your solution package can now deploy binding redirects to your web.config. For example, this means you can have any reference to version 1.0.0.0 redirect to use version 2.0.0.0. Binding redirects are nothing new to .NET, it’s just that SharePoint is now taking advantage of this feature. In fact, SharePoint uses this feature itself to redirect any calls to its DLLs from version 12.0.0.0 to 14.0.0.0. Here is an example of what that might look like.

<Assemblies>
  <Assembly DeploymentTarget="GlobalAssemblyCache" Location="MyWebParts.dll">
    <BindingRedirects>
      <BindingRedirect OldVersion ="1.0.0.0" />
    </BindingRedirects>
    <SafeControls>
      <SafeControl Assembly="MyWebParts, Version=2.0.0.0, Culture=neutral" Namespace="MyWebParts" TypeName="*" Safe="True" />
    </SafeControls>
  </Assembly>
</Assemblies>

I really like this new feature and I think it will make it much easier for developers to version their assemblies.

Checking to see if a list item exists in SharePoint 2010

CoreyRoth — Wed, 21 Oct 2009 16:02:00 GMT

This is another topic that is quite dear to most SharePoint developers. We’ve all been there. We want to know if a list item exists and the indexer is useless. It throws an exception should we attempt to access an item that does not exist. Well as we discovered this week there is a new TryGetList method that works great to determine if a list exists but there really isn’t anything new with a SPBaseCollection. We can do this a number of ways include spmetal.exe to generate strongly typed classes, but I was looking for a solution that didn’t require me to generate any classes. We can now use .OfType<T> to get an IEnumerable<T> that we can use. Once we have IEnumerable<T> it is quite easy to determine if a list item exists using the .Any() method. Look at the example below.

using (SPSite siteCollection = new SPSite("http://moss-server"))
{
    SPList myCustomList = siteCollection.RootWeb.Lists.TryGetList("MyCustomList");
 
    // doesn't throw exception!
    if (myCustomList != null)
    {
        var listEnumeration = myCustomList.Items.OfType<SPListItem>();
        return listEnumeration.Any(p => p.Name == "List Item 1");
    }
}

As you can see using the lambda expression p => p.Name we can do a comparison to see if a list item with that name exists. This will return us a boolean if the item does in fact exist. As you can see this isn't quite as good as a native method that checks for an item that exists, but it does work. If you want to get creative you can also check on other fields such as Title, Id, or whatever. Take a look at this sample.

listEnumeration.Any(p => p["Title"] == "List Item 1");
listEnumeration.Any(p => p.UniqueId == someGuid);

As you can see, LINQ makes it quite easy to determine if an item exists. This may not be the most efficient way to do it as it is rather brute force. It may only be arguably better than using a try / catch block, but it certainly looks cleaner.

Introducing the Visual Web Part in SharePoint 2010

CoreyRoth — Tue, 20 Oct 2009 21:38:19 GMT

You might have seen that there is a new designer for working with web parts in Visual Studio 2010. This is actually, not quite the case, so I thought I would share with you how it actually works. When you use the Visual Web Part SPI (SharePoint Project Item), it will create a number of files for you: a .cs file, a .webpart file, an elements.xml file, and a .ascx file. That’s right, it creates an ASP.NET user control. Now, the Visual Studio interface does a nice job of grouping all of the files together. When you open the designer on the web part, you in fact just open the designer for the user control.

So now you might be thinking, how does this work? Well no SharePoint can’t host user controls directly still. In fact all it does is create a simple web part with a Page.LoadControl() method pointing at our user control. This in fact is the same method we’ve been using for years to get user controls into SharePoint. I don’t really see this as an issue though, since Visual Studio does a great job making it painless to deploy these now and use them inside SharePoint.

Checking to see if a list exists in SharePoint 2010

CoreyRoth — Tue, 20 Oct 2009 20:17:17 GMT

This is a bitter subject with most MOSS 2007 developers because the most common way to do this is by using a try/catch block when you try to use the indexer on SPListCollection. Well, I am pleased to tell you about a new method I discovered on SPListCollection that really made my day. The new TryGetList method takes the name of a list and will get this, return a null if the list doesn’t exist. Take a look at this code.

using (SPSite siteCollection = new SPSite("http://moss-server"))
{
    SPList myCustomList = siteCollection.RootWeb.Lists.TryGetList("MyCustomList");
 
    // doesn't throw exception!
    if (myCustomList != null)
    {
        // do something
    }
}

That’s right! You heard me, instead of throwing an exception, you actually get null back when the list doesn’t exist. No more try/catch blocks. No more extension methods to hide those try/catch blocks. Unfortunately, there is nothing on SPBaseCollection to check if an item exists, but I do have a solution, so be looking for that post.

SharePoint 2010: 12 Hive + 2 = 14 Hive

CoreyRoth — Tue, 20 Oct 2009 20:15:16 GMT

If you’re a SharePoint person, you of course have the following path burned into your memory forever.

C:\Program Files\Common Files\Microsoft Shared\Web Server Extension\12

Well pretty soon, you can replace that with:

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14

Apparently, Microsoft thought the number thirteen was unlucky. Now, I know they are trying to rebrand this as the SharePoint root (or something like that), but we all know that is never going to sick, so I’ll just call it the 14 hive. Anyhow, this is the new place you’ll be doing a lot of your work. Although with the new Visual Studio 2010 tools, you’ll find that you won’t need to come to this folder nearly as much. In this post, I thought I would take a brief moment to point out what I noticed in the 14 hive and make any comments as necessary. None of the underlying folders really have changed but a few things have been added.

In the root of the 14 Hive, I noticed three new folders

UserCode – files used to support sandboxed solutions
WebClients – used for the client OM I believe
WebServices – many new .svc files

If you are interested in what assemblies are present in the ISAPI folder, here is a quick list. It sounds like some classes have moved to different DLLs, but I haven't encountered any yet.

Microsoft.BusinessData.dll
Microsoft.Office.DocumentManagment.dll
Microsoft.Office.Excel.Server.Udf.dll
Microsoft.Office.Excel.Server.WebServices.dll
Microsoft.Office.Policy.dll
Microsoft.Office.SecureStoreService.Security.dll
Microsoft.Office.Server.dll
Microsoft.Office.Server.Search.dll
Microsoft.Office.SharePoint.ClientExtensions.dll
Microsoft.Office.UserProfiles.dll
Microsoft.Office.Word.Server.dll
Microsoft.Office.Workflow.Actions.dll
Microsoft.Office.Workflow.Tasks.dll
Microsoft.SharePoint.Client.dll
Microsoft.SharePoint.Runtime.dll
Microsoft.SharePoint.dll
Microsoft.SharePoint.Linq.dll
Microsoft.SharePoint.Portal.dll
Microsoft.SharePoint.Publishing.dll
Microsoft.SharePoint.Search.dll
Microsoft.SharePoint.Search.Extended.Administration.Common.dll
Microsoft.SharePoint.Search.Extended.Administration.dll
Microsoft.SharePoint.Search.Extended.Administration.ResourceStorage.dll
Microsoft.SharePoint.Search.Extended.Administration.Query.dll
Microsoft.SharePoint.Security.dll
Microsoft.SharePoint.Taxonomy.dll
Microsoft.SharePoint.Taxonomy.Intl.dll
Microsoft.SharePoint.Workflow.Actions.dll
Microsoft.Web.CommandUI.dll

I believe all of these are registered as version 14.0.0.0. There is a lot of new things in the hive including features, site templates, etc, but not that many structural changes from what I can tell.

Writing to the Property Bag using CAML in SP2010

CoreyRoth — Tue, 20 Oct 2009 15:51:32 GMT

There are a number of feature improvements in SharePoint 2010. They have made many changes which allow you to do more with features when they are activated or upgraded. One of the new features it the ability to write to the property bag. To do this we just make use of the new PropertyBag element in your elements.xml file. Here is what a file might look like that writes to the property bag. As usual, I’ll point out that features or syntax could change.

<Elements xmnlns="http://schemas.microsoft.com/sharepoint/">
  <PropertyBag ParentType="Web">
    <Property Name="MyCustomProperty1" Value="some value" />
    <Property Name="MyCustomProperty2" Value="4" />
  </PropertyBag>
</Elements>

I will point out that whatever type you initialize the property with, it will become semi-permanent. Currently, this applies for integers and date/time. If you try to save another type in an existing property later, you will end up with a null value being stored. This will save me some time in the future as I have had to write code to do this in the past.