Friday, March 6, 2009

YouTube Integration with Sitecore 6.0

Recently I got a quite interesting task to build an integration between Sitecore CMS and YouTube repository. I finished that task and I'm willing to share my experience with public. I will describe code only partially. Only those parts I consider the most interesting.

We've been thinking about different approaches and have chosen Sitecore Data Provider approach. We decided to have items in CMS that would represent YouTube videos. One of the challenges was to integrate YouTube videos on the fly when an editor wants to add videos from one or other YouTube author.
This data provider works in read-only mode. If you want to extend this solution, you're welcome to do so.
So let's begin....

Templates

I created two templates that will represent root item for YouTube videos and YouTube video item itself. First one is called "YouTube branch" and the other one is "YouTube video".
YouTube branch is going to be a folder for YouTube video items. Unlike usual folders it has a field that determinates videos of what YouTube author to show beneath it.
YouTube video template extends Flash media template and adds two additional fields: Url and Preview. Url is obviously intended for a YouTube video url. Preview is an iframe field that will show us the preview of YouTube video.

Implementation

There is a required list of methods you must implement to get data provider working. Here are those methods:

- GetItemDefinition
- GetItemFields
- GetChildIDs
- GetParentID
- GetItemVersions

I'm not going to describe what they are for. You can find a detailed explanation on SDN.
One method is missing in this list. It's "GetTemplates" method. We can avoid implementing this method if required templates already exist in our data source.
I'm going to use master database as a source for YouTube video items. I created a couple of templates for this approach so that we can skip implementation of the "GetTemplates" method.
Also I made this data provider publish its items by implementing one publishing method:

- GetPublishQueue

This is how constructor looks for the data provider:
public YouTubeDataProvider(string rootTID, string resourceTID, string videoOwnerField, string contentDB)
{
prefix = ToString();
rootTemplateID = rootTID;
resourceTemplateID = resourceTID;
contentDatabase = contentDB;
videoOwnerFieldName = videoOwnerField;
_items = new Hashtable();
}

rootTemplateID contains an ID of YouTube branch template
resourceTemplateID contains an ID of YouTube video template
contentDatabase is a source database for our YouTube provider (it's master database)
videoOwnerFieldName is a field name of YouTube branch template where you specify an owner of YouTube videos.
_items is a temporary cache for our YouTube videos.
Since we need to use some sort of mapping between YouTube videos and Sitecore items, I'm going to use existing functionality of IDTable. That's what we need "prefix" variable for.

Now let's look at implementation of first method:
public override ItemDefinition GetItemDefinition(ID itemId, CallContext context)
{
ItemDefinition itemDef = null;
string itemName = string.Empty;
if (CanProcessYouTubeItem(itemId, context))
{
// method body
}
return itemDef;
}

The first thing we need to think of is how to determine if the item we are getting from data provider is our YouTube item. That's what CanProcessYouTubeItem method for. This method uses IDTable API to find out if an item comes from YouTube rather than SQL database. Here is the code for the method:
bool CanProcessYouTubeItem(ID id, CallContext context)
{
if (IDTable.GetKeys(prefix, id).Length > 0)
{
return true;
}
return false;
}

The same technic is used for GetItemFields, GetParentID and GetItemVersions methods. For GetChildIDs we had to use different code because we had to work with parent item. Here is a code for the method:
public override Sitecore.Collections.IDList GetChildIDs(ItemDefinition itemDefinition, CallContext context)
{
if (CanProcessParent(itemDefinition.ID, context))
{
// method body
}
return null;
}

CanProcessParent method checks if the item is our YouTube branch item. If it indicates that it is then we start processing our child items that are YouTube videos.
This is the code of the method:
bool CanProcessParent(ID id, CallContext context)
{
Item item = ContentDB.Items[id];
bool canProduce = false;
if (item != null && (ID.Parse(rootTemplateID) == item.TemplateID))
{
canProduce = true;
}
return canProduce;
}

After we get our items, we have to fill out their fields to finish our integration process. Information from YouTube video is mapped in GetItemFields method. This is the code:
public override FieldList GetItemFields(ItemDefinition itemDefinition, VersionUri versionUri, CallContext context)
{
FieldList fields = new FieldList();
if (CanProcessYouTubeItem(itemDefinition.ID, context))
{
string originalID = GetOriginalRecordID(itemDefinition.ID);
TemplateItem templateItem = ContentDB.Templates[resourceTemplateID];
if (templateItem != null)
{
YTItemInfo ytItemInfo = GetYTItemInfo(itemDefinition.ID);
if (ytItemInfo != null)
{
foreach (var field in templateItem.Fields)
{
fields.Add(field.ID, GetFieldValue(field, ytItemInfo));
}
}
}
}
return fields;
}

As you can see it's simply iterates through template fields and maps data from YouTube Entry to Sitecore YouTube video item.
We got YouTube video items in our content tree. But we cannot do anything with them on front-end. One option is to add our data provider to web database to get it working on front-end. But what if we make main data provider know about our items and just publish them as a real items. All I need to do is to get list of IDs of our items in GetPublishQueue method.
This is how the code looks:
public override IDList GetPublishQueue(DateTime from, DateTime to, CallContext context)
{
IDList list = new IDList();
foreach (var item in _items)
{
YTItemInfo ytItemInfo = (YTItemInfo) item;
list.Add(ytItemInfo.ItemID);
}
return list;
}

Since I did not use a publish queue table, YouTube items are going to be published every time to run publish operation on them. You can avoid this if you implement AddToPublishQueue and CleanupPublishQueue methods.

Here is a screen dump the way YouTube item looks in Sitecore media library:

This is a sitecore package that will install data provider on your Sitecore 6.0 CMS. I do not described provider configuration in this article. You can find it in /App_Config/Include folder after installing the package.
NOTE: if installation fails, that means that you run into a known issue. Reinstalling the package should solve it.

7 comments:

Anonymous said...

Hi Ivan,

Thank you very much. Very cool!

If possible, it would be great if you could convert this project into a Sitecore Shared Source module.

http://trac.sitecore.net/Index

Anonymous said...

Whoah, too cool!

Have you done anything (or plan) to make it easier to insert youtube videos on the frontend?

Ivan said...

Thanks guys! I'll try to add it to our trac system as soon as I have time.
Same thing about control that will make it easier to render YouTube video on front-end.

Anonymous said...

Hi Ivan,

I am trying to make a Data Provider that can be hierarchically generated. E.g. I would get a list of CD's from an external data source and then when the user clicks on the logical CD item it can be expanded to show its tracks as Children. We must assume the Tracks need to be fetched Dynamically and separate to the initial CD list. Any idea if this will be viable?

Ivan said...

Sounds as appropriate solution to use data provider.
Here is a mockup for content structure that you will have with your CD data provider:
- CD container (Sitecore item which is a root item for your external data)
--- CD1 item (represents external CD)
------ Track1 (represent CD1's track1)
------ Track2 (represent CD1's track2)
------ .... so forth
--- CD2 item
--- .... so forth

CD items and their tracks will be fetched dynamically from your external source.

Anonymous said...

Yes that structure is what I want to achieve.
I know we can populate a single level of Data Provider driven items but how would we go about Fetching tracks when the CD parent is Expanded?

Ivan said...

When you expand a branch in content tree the data provider pulls out data of item's children and grand-children. It's two levels down from the branch root item. The GetChildIDs method is called for branch root item and for every single children of it.
To pull out external data for two levels, you need to build logic that fetches data depending on what item called the GetChildIDs method.
For example (somewhere inside GetChildIDs method):
......
if (itemDefinition.ID == RootID)
{
//Get list of CD items
}
else if (itemDefinition.TemplateID == CDTemplateID)
{
// Get list of tracks for current CD item
}
......