FC.MetadataExtractor released for SharePoint 2010

A few days ago we released our first product for the new SharePoint 2010 platform: FC.MetadataExtractor 3.0

By far the biggest improvement in SharePoint 2010, in regards to this new release of our product, is the ability to better control the asynchronous ItemAdded event receiver. Given the positive feedback at the SharePoint 2010 conference when Microsoft announced the addition of such a synchronous ItemAdded event, quite a number of people must have encountered that problem.

Recap: in SharePoint 2007, the ItemAdded event fires asynchonously and so does the thread that brings up the EditForm.aspx with the properties of the newly created item. This of course creates a collision if both threads then try to modify that same item. The only way out of this issue was to provide a custom EditForm.aspx page that would wait for the ItemAdded event receiver to finish before the EditForm.aspx would load and access the item.

In SharePoint 2010 the ItemAdded event receiver can now be configured to either fire synchronously or asynchronously and also be given a sequence number. This eliminates the need for a custom EditForm and any manual synchronization and allowed a much "cleaner" solution with the new product release.

On the user interface front, the Ribbon does take a bit to get used to, both for programmers and users. I do think though that accessing the Ribbon is easier for many users than having to search through several popup menus for a certain command.

FC.MetadataExtractor for SharePoint 2010

The ability to have client-side ECMAscript code and the new Notification elements in the user interface is definitely a great step forward. We made use of it for the manual metadata extraction that now happens asynchronously at the server while the user can continue to work in the browser.

Published: Jun-24-10 | 0 Comments | Link to this post

Crawled Property problems

The SharePoint technical documentation leaves a lot to wish for in general, but Crawled Properties is probably one of the worst topics when it comes to documentation. On top of that, the functionality to implement is weak as well. Let me give you some examples:

When you google for "SharePoint CrawledProperty" you get a whopping 1500 search results (I'm not sarcastic, just comparing Google to Bing, read on!) and on Bing you find 122 results on the same search, with the majority being people asking for help, I'd say.

Anne Stenberg provides a few clues on her blog about how to "interpret" the Crawled Properties. Let me quote:

"Property Set ID - A GUID that identifies the property set for the crawled property. Doing a search for the GUID B725F130-47EF-101A-A5F1-02608C9EEBAC and filtering the results for the Property Name of 12 yields several links to related content. One such link, on MSDN, provides a tremendous amount of information. This tells us that this property set is a System property and the propID of 12 is the file size"

While helpful to know, for me it simply shows how awkward the whole naming system is around Crawled Properties. But let's move on and get productive. We want to make things work here...

I'll use a real-world scenario to describe a typical problem around Crawled properties and how to solve it: assume you want to search on the "Date Picture Taken" column of the Picture library. After you put a fresh MOSS installation on a server, you can look up in the Search Administration of the Central Administration what Managed Property is available for that kind of search (only Managed Properties can be used in the search!). You'll find one entry:

A Managed Property (DatePictureTaken) already exists and has a mapped Crawled property called "ows_Image_x0020_CreateDate". While this looks good at first glance it will actually not help us at all. Why? Because the internal name for the "Date Picture Taken" column in the Picture library is "ImageCreateDate", so the CrawledProperty we'd need would be "ows_ImageCreateDate".

Just a quick explanation on the Crawled property naming convention: Crawled Properties are created automatically by the search crawler when it detects new columns that have a value. SharePoint columns always start with "ows_" (probably stands from "Office web server") concatenated with the internal name of the column.

So, let's have a look if there is another Crawled Property available with the right name. Under the Crawled Properties section we find this:

Initial ImageCreateDate Crawled Properties

Hmm, there are even three such Crawled Properties: two with the same, wrong name (including the text/hex version of a blank in the name) and one with the right name... but it's got the wrong type: we need a DateTime field, not a Text field!

OK, time to upload some images that include metadata for the "Date Picture Taken" column. A (shameless ;-) plug for our FC.MetadataExtractor product here: it extracts that DatePictureTaken metadata (and more) by default, and puts it in the right column. After that, starting a Crawl will "nudge" the search crawler to recognize the Date Picture Taken column in the Picture library, because it now has data. And voila, we have the "Date Picture Taken" column linked to the right Crawled Property and in turn we can go ahead and link this Crawled Property to the existing "DatePictureTaken" Managed Property (done in the following screenshot with our FC.ImageSearch product, but you can also do that in the Search Administration section of the Central Administration).

The right DatePictureTaken Crawled Property

But we're still not done: if we'd do a search now we'd get a really unfriendly error from the search service:

This is because we now have two Crawled Properties mapped to the Managed Property, but it can only take input from one at a time, so we need to change another setting for the Managed Property to "include values from a single crawled property...":

Yet another full crawl will help to push the Crawled Property data on to the newly linked Managed Property.

Now comes the fun! In the Search Configuration page of the FC.ImageSearch you can simply add the DatePictureTaken Managed Property that is now linked to the library's "Date Picture Taken" column and add it to the properties for the Advanced Search page:

Now, in the Advanced Search page you can select the "All Images" scope of the FC.ImageSearch solution and then choose from its Managed Properties the DatePictureTaken property:

...and that will get the successful search on our "Date Picture Taken" column:

Well, I still wonder why there are - out of the box - three ambiguous ImageCreateDate Crawled Properties. And what's more, deleting Crawled Properties is not possible in SharePoint, so you'll just have to ignore them.

But at least we got the search to work!

Published: May-12-10 | 4 Comments | Link to this post

Next Product up: FC.ImageSearchResults

Right on the heels of our FC.MetadataExtractor 2.0 release shortly before Christmas we're working hard on getting a fun product out the door: FC.ImageSearchResults. It'll probably be another couple of weeks before we hit beta 1, but I thought I should already mention it here.

All the metadata that can be unleashed by the FC.MetadataExtractor product will be available to search on if the FC.ImageSearchResults solution is installed. And what's best, the solution comes with a custom searchresult page that displays those images nicely with thumbnails and metadata that makes sense for images. I'll blog more about this as we get closer to the public beta release.

As always with our product efforts we're aiming to support both MOSS (the commercial product) as well as WSS 3. This time I am especially happy to tell that there will also be support for a free version of SharePoint: while search in WSS 3 simply sucks, the Search Server Express 2008 (SSX) product (that simply can be installed "over" an existing WSS 3 installation) has all the bells and whistles to serve as the basis for our search solution.

If you run WSS 3 and haven't looked at SSX yet ...I highly recommend you do so ASAP!

The object model of the libraries that get installed with the SSX "upgrade" for WSS 3 have everything we need for our solution. Even the Search user interface that is added to the Central Administration is quite nice. However, there was one thing that was missing: a link to the page to manage the so-called "Shared Scopes". Our solution installs a Share Scope called "All Images" that allows to search across the entire farm and while we do that programmatically we of course prefer it to be visible to the administrator, so what is installed is transparent.

Well, the respective admin page for managing the scopes is actually installed. Just the link is missing. So you can get to that page simply using this link:

http://<server name>:<central admin port>/ssp/admin/_layouts/viewscopesssp.aspx?mode=ssp

Published: Jan-16-10 | 0 Comments | Link to this post

SharePoint DAM: Versioning Metadata

As we've just released the 2.0 version of our FC.MetadataExtractor product I thought I should catch up with some blog postings on DAM functionality in SharePoint.

One of the great things that come with SharePoint 2007 and WSS 3 is that you can keep track not only of the different versions of an image but also of its metadata. This is extremely helpful as entering metadata in a corporate environment is usually done over multiple rounds and maybe even involves several contributors.

All you need to do to activate versioning of metadata is to enable Versioning in the Site Settings:

Once that is enabled you can either create a new version by uploading a new version of the image which includes new metadata, or by editing the metadata directly in SharePoint.

The Version history is now available via the item's context menu and the metadata trail can be seen in the screenshot below:

This is just one example of the great base-level functionality SharePoint 2007 / WSS 3 include that can be applied in many scenarios, but is especially useful for metadata-heavy applications like DAM.

Published: Dec-23-09 | 0 Comments | Link to this post

New SharePoint 2010 beta install

Last night, I read that the beta version of SharePoint 2010 was available on msdn.com and couldn't resist getting an immediate feel for where exactly Microsoft stands with this new release. So I downloaded the SharePoint Foundation 2010 (correlates to the "Windows SharePoint Services", the "free" basic version of SharePoint) and started the install.

SharePoint now has a Welcome dialog that looks familiar to many of the other server products released in the last year or so, including valuable links and - most important! - a link to install the "prerequisites". Usually running this tool means that the install doesn't crash out halfway through when it recognizes that a vital component is missing.

The installation of the prerequisites took about 15 minutes on my machine (that was up-to-date on service packs etc.) which is not too bad. However I got a strange feeling from the fact that configuring the IIS role took about 10 minutes of those 15. Well, it all seemed to have worked in the end.

Onto installing SharePoint - only to be greeted by another alert:

Great, the software is telling me that I made a mistake and that I should have read the note about the installation prerequisites... well, you are welcome to check the first screenshot above for yourself for a mention of that KB97183! BTW, you can grab that Hotfix here.

I'd call that a "departments out of sync error", as this is about a release that was just dropped, which practically means that one team involved with documentation and another team involved in coding the prerequisite checks don't properly communicate. That, however reminds me too much about one of the fatal flaws in SharePoint 2007: obviously the developers never really got their communication right with the technical documentation folks. I was promised in Vegas at the recent SharePoint conference that this time around the documentation would be much better and "complete". Let's see, shall we?

Once past the common License Terms confirmation, the familar SharePoint setup dialogs appeared... slightly different background, but very much the same as in 2007.

This time I opted for something I usually NEVER do: I chose the "Standalone" option, which doesn't allow you to easily add servers to the farm. My reasoning was that, if anything would, this option should work. (I'll try the "real" farm installation right after this)

The install went fine all through-out. However, when I came to the end, the "Applying Updates" status stayed on forever ...as it was late I just went to bed and left it running, so I have no idea what major part was downloaded and replaced at that point.

This morning, the install had completed properly and I was able to start the usual Wizard.

And finally, after the wizard had finished, I got to see the Home page of SharePoint 2010! :-)

Here, of course, I couldn't resist immediately checking the compatibility of one of our SharePoint 2007 products. So I installed the latest beta of our up-coming Metadata Extractor program and voila, the install and deployment worked all fine.

The only issues I could see are with some user interface elements that our solution adds which are not displayed in SharePoint 2010. So my initial feel is that there is a good amount of backwards compatibility, but definitely some additional work required to port SharePoint 2007 solutions to SharePoint 2010.

Published: Nov-18-09 | 0 Comments | Link to this post

ChangedProperties Property - Documentation Error

We're getting ready to ship the 2.0 version of our "Metadata Extractor" product that extracts metadata out of images and copies it into the list columns. One of the cool new features is that it also extracts the metadata when a new version of a file is uploaded (previously it only did that on the initial upload, which was hard enough, given the problems around the asynchronous ItemAdded eventhandler ...but that's another story ;-).

The best way to check what exactly has been changed on an item is to use the ItemUpdating and ItemUpdated events. However when we looked into storing information that was available in the ItemUpdating event that occurs first, so that we could access it in the ItemUpdated afterwards we were a bit confused about the documentation, especially the meaning and usage of the ChangedProperties property. MSDN states that it "gets properties that have changed". Well, not only is that as minimalistic as can be, but also simply wrong:

The ChangedProperties property can actually be used to store values from the ItemUpdating event to access them in the following ItemUpdated eventhandler.

Example:

If you add the following statement in your ItemUpdating eventhandler:

properties.AfterProperties.ChangedProperties.Add("FILEVERSION", item.File.Versions.Count.ToString());

...then you can access this value in the ItemUpdated eventhandler via:

string previousVersionNumber = properties.AfterProperties["FILEVERSION"].ToString();

Btw, if you step through your code you'll see that the key/value pair is actually being added to the ChangedProperties property. SharePoint then copies it over to the AfterProperties collection before the ItemUpdated eventhandler is invoked.

Published: Nov-15-09 | 0 Comments | Link to this post

Custom List Forms with Code-behind

There are a lot of articles on the web about manually modifying the list forms and the ListFormWebPart in particular, but there isn't that much information about modifying these forms programmatically, and the problems that come with that.

To start with, there's a great article on SharePoint List Forms by Hristo Pavlov that I highly recommend.

Customizing the ListFormWebPart is a chapter by itself, but what we needed was to customize the EditForm.aspx. Actually, we needed to inject a control that would prevent the form to come up while the asynchronous "ItemAdded" event, triggered by a file upload to the library, was still busy retrieving metadata (also an adventure in itself, as I consider that clearly a conceptual bug in SharePoint), which would result in an exception and a failure of the upload and/or the metadata extraction.

Now, there are actually two levels of forms one could utilize: each list/library has a set of these forms (New, Display, Edit) and the content types that are associated with the list/library also have these forms.

We chose to replace the EditForm.aspx of the document-type content types with a custom aspx form with code-behind. Why code-behind? Well, there are actually a couple of things one needs to adjust on the fly to make this work. One example would be that a custom application page should reside in the "_layouts/" space and the default.master masterpage that is used by the list forms is not directly accessible from there. But the most important reason was that the ListFormWebPart requires the "ListName" (it's actually the Guid ID of the list) and the "ListItemId" (which is what it sounds like :-) to be set properly when the OnInit event fires.

So, where's the problem? The ListFormWebPart is an incredibly flexible piece of programming art that can render itself on the fly automatically, depending on what list, content type and columns are to be displayed, BUT it does not retrieve the ListName and ListItemId from the Page.REQUEST automatically. Alright, we can do that with the code-behind, but that's NOT the problem ;-) The problem is that the control isn't available during the OnPreInit event to set these properties, and before the OnInit event handler gets fired SharePoint already throws an exception stating that no item could be found for the list and itemId specified ...Duhh!

Well, the solution is actually (as most things in life) trivial: the OnInit event fires on the controls before it fires for the page. Therefore it is important to add a code-behind event handler not only at the page level but also for the Web Part itself:

WebPart OnInit event handler registration

And the rather simple code-behind for this looks like this:


        protected void Lfwp_OnInit(object sender, EventArgs e)
        {
            string listId = Page.Request.QueryString["List"];
            string itemId = Page.Request.QueryString["ID"];

            ListFormWebPart lfwp = (ListFormWebPart)sender;
            lfwp.ListName = listId;
            lfwp.ListItemId = Convert.ToInt32(itemId);
        }

The beauty of this solution is that the same custom EditForm.aspx can then be used not only for all Picture Libraries, but also for Document Libraries.

Published: Oct-12-09 | 8 Comments | Link to this post

Guidelines for Handling Image Metadata

Determining the right taxonomy and picking the right standards to store metadata has long been a challenge for DAM users.

But there's hope! The Metadata Working Group - consisting of heavyweights like Adobe, Apple, Microsoft, Canon and Sony - recently wrapped up a first major draft of their Guidelines for Handling Image Metadata.

So, if you're still bedazzled about what to do about tagging and categorizing your images, there's a document for you to read!

If you're in the L.A. area on November 10-11, you might also want to attend the annual Henry Stewart DAM event. Andrew Salop, a world expert on XMP metadata, will be discussing these guidelines.

Published: Nov-05-08 | 0 Comments | Link to this post