Working with Lucene Search Index in Sitecore 6. Part I - Why/When to use

How often did you stress yourself thinking on the question "What's the most efficient way to retrieve Sitecore items?"?

Here are possible ways to do it:

• Sitecore API
• Sitecore Query
• SQL custom stored procedures
• Lucene index

Let's take a look at each option briefly.

Sitecore API - the most popular way to get an item. All you need is an item ID and simple call of GetItem() method of database instance will get you the latest item version in current language. If you want an item in specific language, not a problem: GetItem(, ). Want specify version, not a problem either: GetItem(, , ). There are many ways to get an item through Sitecore API. My point is that it's the most easiest way to do it.

Sitecore Query - easy way to get a bunch of items filtered by some criteria. Build a string query, which kinda look like XPath query, and run it on a database instance. Read more about Sitecore Query.

SQL custom stored procedures - we all know how to create a SQL stored procedure. Connections strings already exist in Sitecore solution. All you need is to use some SQL management studio to create a stored procedure for the database. The question is why would you do it if API already exists? It makes sense only if you run complex query with lots of filtering conditions so that SQL will return you only items that you're searching for.

Lucene index - why would I use separate data storage/data index to get an item from Sitecore database? It does not make sense. Agree, for an item it does not but for a collection of items it has a perfect sense to do it. Why? Maybe because performance is much better when you are working with big amount of data. Moreover that's a search index. Which means that data are perfectly organized to be searched.

Let's talk about performance characteristics for each of these options and compare them.

Sitecore API - uses Sitecore data provider to pull out information from the database. If item that we're looking for is cached the call to SQL database is avoided and item gets retrieved from one of Sitecore caches. Using Sitecore caches gives you huge performance benefit. That's why we always worried about caches configuration in Sitecore solution. What if you need to get a number of items by some criteria, let's say template ID and belonging to some category (from here on I'm gonna use this condition for all item retrieval options). You run foreach or for or any other logic that goes through the content tree and checks for a TemplateID property and category field of every single item. If item resides in cache, it will be quick enough but still the code will have to request every single item from the content tree.

Sitecore Query - the picture with Sitecore Query is very similar to Sitecore API but it get's more slower when number of items is growing. Kim has a good article that explains Sitecore Query performance.

NOTE: I'm not talking about fast Sitecore Query introduced in Sitecore 6.

SQL stored procedures - it seems to be a good approach to go with if you're searching for items through the content tree. The thing is there will be only one stored procedure that executes query by specified criteria. Also you will have to consider caching option for retrieved items if you have very deep content tree and items of some specific time are not gathered under one branch. My point is that it can cost you more then the benefits that you will get. I would go with this approach only if I cannot do it with Lucene resources.

Lucene index - again it's search index. It perfectly fits searching options. When you search for items with specified criteria it neither requests an item instance nor runs query over database tables. It goes through the fields that you have in your index and compares their values to your search conditions. The only thing if search query uses custom fields to filter data by, the fields must be added to your index. Otherwise you will get zero results. It's very easy to do though. I will describe it in the next part of the article.

Conclusion: Let's answer those questions in the topic of this part.

Why to use lucene index?

The approach is one of the most (maybe event the best) efficient ones. Search is quick enough that in most cases you don't even need to implement custom caching for the retrieved data.

When to use lucene index?

When you need to run search queries with specific criteria on huge number of data.

Don't forget the rule - the more complex search query gets the slower it works ;).

In next part we will look into Lucene search index introduced in Sitecore 6.

Information about "old" Lucene index (it was introduced in Sitecore 5) you can find here.

Restrict access to advanced media upload options

Sometimes you want to give an access to Advanced Media Upload for editors. When you do it, they get access to all the advanced options that you might not want to share with them.

This package allows you to configure access to advanced media upload options by tighten security on Sitecore items.

After installing the package, you can configure advanced options access for presets items at /sitecore/System/Settings/Media/Presets path.

Download the package.

YouTube Integration source code

For those who's interested, the source code for YouTube data provider is available on Sitecore Trac system. 

Enjoy!

YouTube Integration with Sitecore 6.0

Recently I got a quite interesting task to build an integration between Sitecore CMS and YouTube repository. I finished that task and I'm willing to share my experience with public. I will describe code only partially. Only those parts I consider the most interesting.

We've been thinking about different approaches and have chosen Sitecore Data Provider approach. We decided to have items in CMS that would represent YouTube videos. One of the challenges was to integrate YouTube videos on the fly when an editor wants to add videos from one or other YouTube author.
This data provider works in read-only mode. If you want to extend this solution, you're welcome to do so.
So let's begin....

Templates

I created two templates that will represent root item for YouTube videos and YouTube video item itself. First one is called "YouTube branch" and the other one is "YouTube video".
YouTube branch is going to be a folder for YouTube video items. Unlike usual folders it has a field that determinates videos of what YouTube author to show beneath it.
YouTube video template extends Flash media template and adds two additional fields: Url and Preview. Url is obviously intended for a YouTube video url. Preview is an iframe field that will show us the preview of YouTube video.

Implementation

There is a required list of methods you must implement to get data provider working. Here are those methods:

- GetItemDefinition
- GetItemFields
- GetChildIDs
- GetParentID
- GetItemVersions

I'm not going to describe what they are for. You can find a detailed explanation on SDN.
One method is missing in this list. It's "GetTemplates" method. We can avoid implementing this method if required templates already exist in our data source.
I'm going to use master database as a source for YouTube video items. I created a couple of templates for this approach so that we can skip implementation of the "GetTemplates" method.
Also I made this data provider publish its items by implementing one publishing method:

- GetPublishQueue

This is how constructor looks for the data provider:
public YouTubeDataProvider(string rootTID, string resourceTID, string videoOwnerField, string contentDB)
{
prefix = ToString();
rootTemplateID = rootTID;
resourceTemplateID = resourceTID;
contentDatabase = contentDB;
videoOwnerFieldName = videoOwnerField;
_items = new Hashtable();
}

rootTemplateID contains an ID of YouTube branch template
resourceTemplateID contains an ID of YouTube video template
contentDatabase is a source database for our YouTube provider (it's master database)
videoOwnerFieldName is a field name of YouTube branch template where you specify an owner of YouTube videos.
_items is a temporary cache for our YouTube videos.
Since we need to use some sort of mapping between YouTube videos and Sitecore items, I'm going to use existing functionality of IDTable. That's what we need "prefix" variable for.

Now let's look at implementation of first method:
public override ItemDefinition GetItemDefinition(ID itemId, CallContext context)
{
ItemDefinition itemDef = null;
string itemName = string.Empty;
if (CanProcessYouTubeItem(itemId, context))
{
// method body
}
return itemDef;
}

The first thing we need to think of is how to determine if the item we are getting from data provider is our YouTube item. That's what CanProcessYouTubeItem method for. This method uses IDTable API to find out if an item comes from YouTube rather than SQL database. Here is the code for the method:
bool CanProcessYouTubeItem(ID id, CallContext context)
{
if (IDTable.GetKeys(prefix, id).Length > 0)
{
return true;
}
return false;
}

The same technic is used for GetItemFields, GetParentID and GetItemVersions methods. For GetChildIDs we had to use different code because we had to work with parent item. Here is a code for the method:
public override Sitecore.Collections.IDList GetChildIDs(ItemDefinition itemDefinition, CallContext context)
{
if (CanProcessParent(itemDefinition.ID, context))
{
// method body
}
return null;
}

CanProcessParent method checks if the item is our YouTube branch item. If it indicates that it is then we start processing our child items that are YouTube videos.
This is the code of the method:
bool CanProcessParent(ID id, CallContext context)
{
Item item = ContentDB.Items[id];
bool canProduce = false;
if (item != null && (ID.Parse(rootTemplateID) == item.TemplateID))
{
canProduce = true;
}
return canProduce;
}

After we get our items, we have to fill out their fields to finish our integration process. Information from YouTube video is mapped in GetItemFields method. This is the code:
public override FieldList GetItemFields(ItemDefinition itemDefinition, VersionUri versionUri, CallContext context)
{
FieldList fields = new FieldList();
if (CanProcessYouTubeItem(itemDefinition.ID, context))
{
string originalID = GetOriginalRecordID(itemDefinition.ID);
TemplateItem templateItem = ContentDB.Templates[resourceTemplateID];
if (templateItem != null)
{
YTItemInfo ytItemInfo = GetYTItemInfo(itemDefinition.ID);
if (ytItemInfo != null)
{
foreach (var field in templateItem.Fields)
{
fields.Add(field.ID, GetFieldValue(field, ytItemInfo));
}
}
}
}
return fields;
}

As you can see it's simply iterates through template fields and maps data from YouTube Entry to Sitecore YouTube video item.
We got YouTube video items in our content tree. But we cannot do anything with them on front-end. One option is to add our data provider to web database to get it working on front-end. But what if we make main data provider know about our items and just publish them as a real items. All I need to do is to get list of IDs of our items in GetPublishQueue method.
This is how the code looks:
public override IDList GetPublishQueue(DateTime from, DateTime to, CallContext context)
{
IDList list = new IDList();
foreach (var item in _items)
{
YTItemInfo ytItemInfo = (YTItemInfo) item;
list.Add(ytItemInfo.ItemID);
}
return list;
}

Since I did not use a publish queue table, YouTube items are going to be published every time to run publish operation on them. You can avoid this if you implement AddToPublishQueue and CleanupPublishQueue methods.

Here is a screen dump the way YouTube item looks in Sitecore media library:

This is a sitecore package that will install data provider on your Sitecore 6.0 CMS. I do not described provider configuration in this article. You can find it in /App_Config/Include folder after installing the package.
NOTE: if installation fails, that means that you run into a known issue. Reinstalling the package should solve it.

Adding buttons to inline-editing frame

Today's post is dedicated to Sitecore inline-editor and some customization you can put into it. The question I'm going to address is:
"Is it possible to add custom buttons to inline-editor frame?"
The answer is simple as is: "Yes, it is" :). Wait... don't go away. Read further to find out the solution.

Our challenge is to add a button to inline-editor. Let's do that for Rich Text field.
I put this approach into the steps to outline this process. Let's start:

1) Open Sitecore Client and go to the Core database.
As you know (or maybe you're going to find out right away) Rich Text field has several profiles to give different sets of functionality to logged in users. Let's go to default profile and check its configuration, here is the path in content tree: "/sitecore/system/Settings/Html Editor Profiles/Rich Text Default".

2) The configuration node we're interested in is "WebEdit Buttons". Go ahead and expand it.
Beneath the node you can see all available buttons which are going to show up as soon as you put mouse cursor over the rich text data in Page Editor.

3) Since we're already here let's add our own button to the current set. A button should be created from "/sitecore/templates/System/WebEdit/WebEdit Button" template.

Header field contains the name of the button in inline-editor frame.
Icon field has the icon for the button
Click field is the most important part. It has command message that should be sent when you hit the button.

a) Here is the format of this field for the custom command: "javascript:return Sitecore.WebEdit.editControl($JavascriptParameters, "webedit:mybutton:message")"
Javascript command is required to set you command message that is passed as second parameter for the "editControl" function.
"webedit:mybutton:message" is command message. Don't be concerned about the format of this message. It can have any format you want. For instance: "webedit:mymessage" or "webedit:mybutton:mymessage". It's good when the first part of the message describes an area the message belongs to, e.g. "webedit:".

b) Here is another example to run JavaScript function through the button.
Click field should look like this: javascript:Sitecore.WebEdit.execute("js_function_name", true, "parameter_for_js_function");
This example changes the font size:
javascript:Sitecore.WebEdit.execute("fontsize", true, "20");

NOTE: if you use javascript function, the Header field for the button item should be left empty.

Tooltip field describes itself :).

4) This is the final step. You have to create a command handler that will do action you want.
I'm not going to touch command creation process here. You can find sample in John's article here.
NOTE: webedit command handler class must be inherited from Sitecore.Shell.Applications.WebEdit.Commands.WebEditCommand one.