Showing posts with label Sitecore 6. Show all posts
Showing posts with label Sitecore 6. Show all posts

Friday, November 23, 2012

Dynamic Workflow module

I recently published a new module to our Sitecore Marketplace site. It’s called Dynamic Workflow.

The gist of this module is to help content administrators to easily configure workflows of any complexity level. It takes advantage of Rules Engine to execute workflow actions that contain set of rules. Any workflow command or state should have a workflow action that determines what operations should be executed on the item.

The module comes with some basic rule actions and could be easily extended with custom ones.

Here are a few basic scenarios that are very easy to solve with the module:

  • Set up a landing workflow that decides which workflow should be applied to a specific type of item.
  • Configure an action that moves an item to a specific location. For instance, all media assets get uploaded into a common folder and workflow sorts them out based on an attribute.
  • Skip irrelevant workflow states based on some criteria, e.g. user role.
  • Add a custom validation rule that controls application of mandatory data, etc.

I’d be happy to hear any feedback or ideas for improvements from the community.

Thursday, September 20, 2012

Interacting with Content Editor in Sitecore 6

I recently stumbled across an interesting challenge. The issue was to programmatically select an item from within a custom item editor tab.
In Sitecore 5.x it was quite easy to do. All you need to code was this.
Sitecore.Context.ClientPage.SendMessage(this, string.Format("load:item(id={0})", itemId));
Quite simple, huh?

In Sitecore 6 some revamping has been done to Content Editor and now aforementioned approach does not work. The “SendMessage” method does not broadcast the message called from the custom item editor all the way to parental window of Content Editor. The message is sent to an object of custom editor and gets handled by it.

Here is the way it should be done in Sitecore 6.
string msg = string.Format("window.parent.scForm.postRequest(\"\",\"\",\"\",\"item:load(id={0})\")", itemId);
Sitecore.Web.UI.Sheer.SheerResponse.Eval(msg);
Note: “window.parent” in this message is mandatory. Otherwise it won’t work.

P.S. Saving brain-power energy for whomever stumbles upon.

Monday, November 28, 2011

Rich Text auto-save

Another customization to address rising demand for auto save functionality in Rich Text fields. Per teammate’s suggestion I started with an approach that we found in our knowledge base system. I expected that some customization would be required as it was created a while back and we updated Rich Text editor since then. Here is what we’ve done to achieve the goal. The customization introduces a few changes to the following files at /sitecore/shell/Controls/Rich Text Editor folder:
  • EditorPage.aspx – added some JS changes and changed class inheritance to a custom one.
  • EditorPage.js – made some JS changes.
  • EditorWindow.aspx – made some JS changes.
One can use any merger tool to see what changes were made to aforementioned files. I also included a link to Sitecore package that contains required customization.
Keep in mind that the package contains a DLL so that app pool will be recycled during the package installation.
Please BACKUP files listed above prior to installing the package.
Keep in mind that a Sitecore update that overrides any of the listed files will cease auto save functionality for RTE fields.
This feature was tested in Sitecore 6.4.1 Update-3 and Update-5 as well as Sitecore 6.5.0 Update-1.
Sitecore package link.

Friday, September 9, 2011

Advanced Publish Dialog

Every once in a while our customers ask if it’s possible to terminate a publishing job. A demand for this functionality increased as our customers started pouring more and more content into their Sitecore implementations. Sometimes an innocent  publishing job could become vicious and freeze other publications.

In this blog post I’m going to present an approach that allows a user to cancel a triggered publishing job. This became possible as new publishing pipelines were introduced in Sitecore 6.

Main idea of this tool was to add flexibility to control Publish Dialog UI through Sitecore security mechanism. For instance, hide or disable publish options and  “Publish Subitems” checkbox using user security settings. This was implemented by introducing Sitecore setting items for Publish Dialog UI controls. The items located at /sitecore/system/Settings/Publish folder.

The security settings on publish option items as well as checkbox field, Disabled, control appearance and accessibility of the controls on Publish Dialog form.
There are a few more setting items under the /Behaviors branch described below:

  • Publish Cancel Behavior
    Controls publish cancelling mechanism. If “Cancel with exception” field is checked, it throws an exception after publishing job is cancelled. This is necessary to suppress post publish events and prevent update of LastPublish property when needed.
  • Cancel Button
    Controls accessibility of “Cancel” button in Publish Dialog form. By default it has standard behavior – button is disabled after one starts publishing process. When enabled, it’s possible to cancel current publishing job.
  • Confirm subitems publish
    If enabled, it pops-up a confirmation dialog when “Publish Subitems” checkbox is selected. You can tweak the confirmation message at “/sitecore/system/Dictionary/A/Are you sure you want to publish subitems too” dictionary item in the Core database.

There are two versions of this component:

1. v1.1.1 – developed for AppPool running any .NET version

2. v1.2.0 – developed for AppPool running .NET 4.0 version ONLY. It takes advantage of a feature in .NET 4.0 that allows to speed up publishing by assigning multiple threads to publish items.

Setting that controls number of allowed threads for publishing process is located in /App_Config/Include/Sitecore.SharedSource.AdvancedPublishDialog.config file. By default allows 2 publishing threads.

<!--  Max number of concurrent threads for publishing process.
If this setting is not set Environment.ProcessorCount variable will be used.
-->
<setting name="Publishing.MaxConcurrentThreads" value="2"/>



To observe and cancel publishing jobs an additional application was added to the component – Publish Status Manager. A link to this app appears in Publish Dialog form if a user has access to the app. It’s also possible to open the app from Sitecore start button, again, if the user has access to it.


From within the app one can select a job and hit “Cancel” button to terminate it. By clicking “Cancel all”, all publishing jobs will be forced to be finished.


A few words on how publishing cancel works and its consequences.
When one triggers publishing cancellation, the job gets set to Finished state which is utilized in customized ProcessQueue processor of <publish> pipeline to get off the publishing loop. If “Publish Cancel Behavior” is enabled (setting described above), then custom ProcessPublishCancel pipleline will throw an exception to suppress post publish events.


The caveats of cancelling publishing process.


Publishing of each item is an atomic operation. When you cancel the publishing, there is no mechanism to roll back changes for already processed items. The data for already published items will make their way to the publishing target database. The search indexes will index new data as soon as IndexingManager reads new entries from History table. New published data may not appear on the public site if old data sit in HTML cache of presentation controls and “publish:end” event did not rise to clear it out.


The “Cancel with exception” behavior on “Publish Cancel Behavior” setting item can suppress “publish:end” event and leave LastPublish property unchanged. This could be required if one wants to re-publish processed items, which got published before the publishing job was cancelled, at next Incremental publishing.
If this setting is not set, then caches will be cleared for all processed items whenever publishing is terminated. The LastPublish property will be updated with publish cancellation time. Next Incremental publish will pick up items from the point where they were left when publish cancel event got fired.


Having said this, I’d recommend to leave “Cancel with exception” field unchecked unless you have a strong requirement to republish processed items along with those that were skipped by publish cancellation event.


Here as screencast of quick overview of the component:


Advanced Publish Dialog for Sitecore

UPDATED on Sep 20, 2011
Support for v1.1.x was dropped as v1.2.x does not require AppPool to run .NET 4.0. The link to v1.1.x is no longer available.


Below are the links to the mentioned Sitecore packages:
Sitecore package of AdvancedPublishDialog.


If you have any enhancement ideas or additional features for this module, feel free to express them in the comments.
Enjoy!

Tuesday, June 28, 2011

Ensure valid Sitecore Internal Links in Page Editor

(Updated on Dec 9, 2011)
Refer to this post for updated solution.

(Updated on July 27, 2011)

A workaround developed for one of our customers incited me to blog about it as I can see how many other may get affected by this issue.

The issue was that whenever an editor opens RTE control in Page Editor, to get access to all provided functions, and saves the changes, all internal links (that start with “~/link.aspx”) get prefixed with the host name that is used in the browser to access the Page Editor. I could recreate this behavior only in IE browser and only when it runs in compatibility mode. Got same results in both IE8 and IE9 running in compatibility mode. Normal mode of IE8 does not cause the issue.

To address this issue I hooked into <saveUI> pipeline my processor that corrects internal links based on regex that is passed to match a part of the link. Here is the code I came up with:

Code Snippet
  1. using System;
  2. using System.Text.RegularExpressions;
  3. using Sitecore.Configuration;
  4. using Sitecore.Data.Fields;
  5. using Sitecore.Data.Items;
  6. using Sitecore.Pipelines.Save;
  7.  
  8. namespace Sitecore.Support.Pipelines.Save
  9. {
  10.    public class EnsureRichTextRelativeLinks
  11.    {
  12.       public void Process(SaveArgs args)
  13.       {
  14.          if (args.HasSheerUI)
  15.          {
  16.             if ((args.Result == "no") || (args.Result == "undefined"))
  17.             {
  18.                args.AbortPipeline();
  19.             }
  20.             else
  21.             {
  22.                for (int i = 0; i < args.Items.Length; i++)
  23.                {
  24.                   SaveArgs.SaveItem item = args.Items[i];
  25.                   Item contentItem = Context.ContentDatabase.Items[item.ID, item.Language, item.Version];
  26.                   if (contentItem != null)
  27.                   {
  28.                      foreach (SaveArgs.SaveField field in item.Fields)
  29.                      {
  30.                         Field fld = contentItem.Fields[field.ID];
  31.                         if (fld != null && fld.Type.Equals("rich text", StringComparison.InvariantCultureIgnoreCase))
  32.                         {
  33.                            if (!string.IsNullOrEmpty(field.Value))
  34.                            {
  35.                               field.Value = EnsureRelativeLinks(field.Value);
  36.                            }
  37.                         }
  38.                      }
  39.                   }
  40.                }
  41.             }
  42.          }
  43.       }
  44.  
  45.       protected virtual string EnsureRelativeLinks(string fieldValue)
  46.       {
  47.          string internalLinkPattern = Settings.GetSetting("PageEditor.InternalLinkReplacePattern",
  48.                                                           "((http)|(https)):((//)|(\\\\))({0}).*(~/link.aspx)");
  49.          string internalLinkReplacementValue = Settings.GetSetting("PageEditor.InternalLinkReplacementValue",
  50.                                                                    "~/link.aspx");
  51.          string mediaLinkPattern = Settings.GetSetting("PageEditor.MediaLinkReplacePattern",
  52.                                                        "((http)|(https)):((//)|(\\\\))({0}).*(~/media)");
  53.          string mediaLinkReplacementValue = Settings.GetSetting("PageEditor.MediaLinkReplacementValue", "~/media");
  54.          Regex linkPattern = new Regex(string.Format(internalLinkPattern, Sitecore.Web.WebUtil.GetHostName()), RegexOptions.IgnoreCase);
  55.          Regex mediaPattern = new Regex(string.Format(mediaLinkPattern, Sitecore.Web.WebUtil.GetHostName()), RegexOptions.IgnoreCase);
  56.          string value = linkPattern.Replace(fieldValue, internalLinkReplacementValue);
  57.          value = mediaPattern.Replace(value, mediaLinkReplacementValue);
  58.  
  59.          return value;
  60.       }
  61.    }
  62. }

I put regex pattern as well as replacement strings into include config file to make the adjustment easier if necessary. Here is how the config file looks like:

Code Snippet
  1. <configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  2.   <sitecore>
  3.     <processors>
  4.       <saveUI>
  5.         <!--
  6.         Fix to address an issue of internal links being converted to absolute links
  7.         after saving content of RTE field in IE browser running in compatibility mode.
  8.         The fix is developed by Sitecore support and should be removed after the problem is fixed in the core product.
  9.         -->
  10.         <processor mode="on" type="Sitecore.Support.Pipelines.Save.EnsureRichTextRelativeLinks, Sitecore.Support.341310" patch:after="processor[@type='Sitecore.Pipelines.Save.ConvertLayoutField, Sitecore.Kernel']" />
  11.       </saveUI>
  12.     </processors>
  13.     <settings>
  14.       <!-- RegEx pattern to ensure valid internal links during the save event in Page Editor.
  15.            The issue occurs in IE browser running in compatibility mode.
  16.            The pattern will be replaced to a value defined at PageEditor.InternalLinkReplacementValue
  17.            The {0} parameter is used to insert a host name used in the browser to access Page Editor.
  18.       -->
  19.       <setting name="PageEditor.InternalLinkReplacePattern" value="((http)|(https)):((//)|(\\\\))({0}).*(~/link.aspx)" />
  20.       <!-- Replacement string for regex pattern defined in PageEditor.InternalLinkReplacePattern setting.
  21.       -->
  22.       <setting name="PageEditor.InternalLinkReplacementValue" value="~/link.aspx" />
  23.       <!-- RegEx pattern to ensure valid media links during the save event in Page Editor.
  24.            The issue occurs in IE browser running in compatibility mode.
  25.            The pattern will be replaced to a value defined at PageEditor.MediaLinkReplacementValue
  26.            The {0} parameter is used to insert a host name used in the browser to access Page Editor.
  27.       -->
  28.       <setting name="PageEditor.MediaLinkReplacePattern" value="((http)|(https)):((//)|(\\\\))({0}).*(~/media)" />
  29.       <!-- Replacement string for regex pattern defined in PageEditor.MediaLinkReplacePattern setting.
  30.       -->
  31.       <setting name="PageEditor.MediaLinkReplacementValue" value="~/media" />
  32.     </settings>
  33.   </sitecore>
  34. </configuration>

Oh yeah, I wasn’t sure if this could happen to media links (couldn’t reproduce it locally) but decided to add the same replacement functionality for “~/media” links as well. If you find it useless, feel free to remove that part :).

This code was developed and tested in Sitecore 6.4.1 rev.110324. It’s expected to work in any 6.4 version. Can’t see any problems with 6.5 but I haven’t tested it there.

As all the code fit into the snippet boxes above, I don’t provide links to sources for download. Though here is the link to Sitecore package that installs the fix.

Hope it saves you some time!

Tuesday, August 31, 2010

Adding custom fields to the index

In this post I want to show how to address a missing feature that was a part of “old” lucene index implementation. This article will provide an example how one can customize Lucene search configuration so that it’s possible to add custom fields to the index.

First off, let’s create a configuration that would allow us to add additional fields to the indexed data.

<index id="News" type="Sitecore.Search.Index, Sitecore.Kernel">
<
param desc="name">$(id)</param>
<
param desc="folder">_news</param>
<
Analyzer ref="search/analyzer" />
<
locations hint="list:AddCrawler">
<
examples-news type="LuceneExamples.DatabaseCrawler,LuceneExamples">
<
Database>web</Database>
<
Root>/sitecore/content</Root>
<
IndexAllFields>true</IndexAllFields>
<
include hint="list:IncludeTemplate">
<
news>{788EF1BE-B71E-4D59-9276-50519BD4F641}</news>
<
tag>{4DD970FB-2695-4E50-96F3-A766F7D6CAF1}</tag>
</
include>
<
fields hint="raw:AddCustomField">
<
field luceneName="author" storageType="no" indexType="tokenized">__updated by</field>
<
field luceneName="changed" storageType="yes" indexType="untokenized">__updated</field>
</
fields>
</
examples-news>
</
locations>
</
index>


There is a new configuration section in this example. It’s <fields> section that introduces two fields “author” and “changed”. These fields will be added to a fields collection of each indexed item. Basically, there is AddCustomField method that gets called for every <field> configuration entry to identify a custom field that is going to be added to the fields collection.


Description of configuration attributes:


  • luceneName  is a field name that appears in lucene index.
  • storageType  is a storage type for lucene field. It can have the following values:
    • no
    • yes
    • compress
  • indexType  is an index type for lucene field. It can have the following values:
    • no
    • tokenized
    • untokenized
    • nonorms

Refere to Lucene documentation to find out what each of these options mean: store and index.


Now all you need to do is to loop through the collection of custom fields in the overridden AddAllFields method and add them to the indexed data.


I created a custom class called CustomField that helps to manage custom field entries. Below is the example of this class as well as additional methods for extended DatabaseCrawler. Since code for the DatabaseCrawler was already published in this blog post, I’m not going to duplicate it here.


Here is a code for CustomField class.


using System.Xml;
using Sitecore.Data;
using Sitecore.Data.Items;
using Sitecore.Xml;
using Lucene.Net.Documents;

namespace LuceneExamples
{
public class CustomField
{
public CustomField()
{
FieldID = ID.Null;
FieldName = "";
LuceneFieldName = "";
}

public ID FieldID
{
get;
private set;
}

public string FieldName { get; private set; }

public Field.Store StorageType { get; set; }

public Field.Index IndexType { get; set; }

public string LuceneFieldName { get; private set; }

public static CustomField ParseConfigNode(XmlNode node)
{
CustomField field = new CustomField();
string fieldName = XmlUtil.GetValue(node);
if (ID.IsID(fieldName))
{
field.FieldID = ID.Parse(fieldName);
}
else
{
field.FieldName = fieldName;
}
field.LuceneFieldName = XmlUtil.GetAttribute("luceneName", node);
field.StorageType = GetStorageType(node);
field.IndexType = GetIndexType(node);

if (!IsValidField(field))
{
return null;
}

return field;
}

public string GetFieldValue(Item item)
{
if (!ID.IsNullOrEmpty(FieldID))
{
return item[ID.Parse(FieldID)];
}
if(!string.IsNullOrEmpty(FieldName))
{
return item[FieldName];
}
return string.Empty;
}

private static bool IsValidField(CustomField field)
{
if ((!string.IsNullOrEmpty(field.FieldName) || !ID.IsNullOrEmpty(field.FieldID)) && !string.IsNullOrEmpty(field.LuceneFieldName))
{
return true;
}
return false;
}

private static Field.Index GetIndexType(XmlNode node)
{
string indexType = XmlUtil.GetAttribute("indexType", node);
if (!string.IsNullOrEmpty(indexType))
{
switch (indexType.ToLowerInvariant())
{
case "no":
return Field.Index.NO;
case "tokenized":
return Field.Index.TOKENIZED;
case "untokenized":
return Field.Index.UN_TOKENIZED;
case "nonorms":
return Field.Index.NO_NORMS;
}
}
return Field.Index.TOKENIZED;
}

private static Field.Store GetStorageType(XmlNode node)
{
string storage = XmlUtil.GetAttribute("storageType", node);
if (!string.IsNullOrEmpty(storage))
{
switch (storage.ToLowerInvariant())
{
case "no":
return Field.Store.NO;
case "yes":
return Field.Store.YES;
case "compress":
return Field.Store.COMPRESS;
}
}
return Field.Store.NO;
}
}
}


And the code for additional methods for DatabaseCrawler.


/// <summary>
///
Loops through the collection of custom fields and adds them to fields collection of each indexed item.
/// </summary>
/// <param name="document">
Lucene document</param>
/// <param name="item">
Sitecore data item</param>
private void AddCustomFields(Document document, Item item)
{
foreach(CustomField field in _customFields)
{
document.Add(CreateField(field.LuceneFieldName, field.GetFieldValue(item), field.StorageType, field.IndexType, Boost));
}
}

/// <summary>
///
Creates a Lucene field.
/// </summary>
/// <param name="fieldKey">
Field name</param>
/// <param name="fieldValue">
Field value</param>
/// <param name="storeType">
Storage option</param>
/// <param name="indexType">
Index type</param>
/// <param name="boost">
Boosting parameter</param>
/// <returns></returns>
private Fieldable CreateField(string fieldKey, string fieldValue, Field.Store storeType, Field.Index indexType, float boost)
{
Field field = new Field(fieldKey, fieldValue, storeType, indexType);
field.SetBoost(boost);
return field;
}

/// <summary>
///
Parses a configuration entry for a custom field and adds it to a collection of custom fields.
/// </summary>
/// <param name="node">
Configuration entry</param>
public void AddCustomField(XmlNode node)
{
CustomField field = CustomField.ParseConfigNode(node);
if (field == null)
{
throw new InvalidOperationException("Could not parse custom field entry: " + node.OuterXml);
}
_customFields.Add(field);
}


Last thing that is left to do is to call AddCustomFields method from AddAllFields one.


protected override void AddAllFields (Documentdocument, Itemitem, bool versionSpecific)
{
    ………………………………………
    AddCustomFields(document, item);
}


You can take it even further and add support for some field interpreter for each field configuration entry.


Hope you'll find it useful.

Tuesday, August 3, 2010

Language filtered Multilist field

Recently I happened to help one of our clients to create a custom Multilist field that gives you selecting options only if item has  at least one translated version in a current content language. From content author’s point of view it makes a lot of sense. Why to give them an option that is not useful?

It’s being a while since I created a custom field that has to take into account content language selection. The main challenge for me was to retrieve that content language. I did remember that there is a property that the Content Editor (CE) sets at run-time for every field but could not recall its name. So after several minutes of browsing my code storage of all samples for all Sitecore versions, I finally found it. So, here are the properties that you need to define in your custom field if you are planning to use them afterwards:

- ItemLanauage – represents a content language selected in the CE.

- ItemID – contains the item ID the field belongs to.

- ItemVersion – contains selected item version.

- ReadOnly – indicates if field is a readonly. If it is, then it will be grayed out and not editable.

- Source – represents the source value from field definition on a data template.

- FieldID – contains the field ID.

In my example I needed only ItemLanguage property. When I put things together the code looked like this:

using Sitecore.Shell.Applications.ContentEditor;
using Sitecore.Data.Items;

namespace Sitecore.Shell.Applications.ContentEditor.CustomExtensions
{
public class LanguageFilteredMultilist : Sitecore.Shell.Applications.ContentEditor.MultilistEx
{
#region Overrides

protected override Item[] GetItems(Item current)
{
Item[] items = base.GetItems(current);

var filteredItems = items.Where(item =>
{
string lang = ItemLanguage;
if (!string.IsNullOrEmpty(lang))
{
var versions = Sitecore.Data.Managers.ItemManager.GetVersions(item, Sitecore.Data.Managers.LanguageManager.GetLanguage(lang, item.Database));
return versions.Count > 0;
}
return false;
}
);

return filteredItems.ToArray<Item>();
}

#endregion Overrides

// Content Editor sets this property. It has a content language from the Content Editor.
public string ItemLanguage
{
get;
set;
}
}
}


Simple enough isn’t it. Don’t forget to add your custom field to /App_Config/FieldTypes.config file if it’s supposed to contain references to other items. In my case it should be added since it’s a multilist. If you forget to do it, the LinkDatabase won’t update references for your field.



That’s all for now.

Wednesday, November 25, 2009

Working with Lucene Search Index in Sitecore 6. Part III - Code examples

If you read my previous posts about Lucene search index, then you already know how to configure it and how it works in Sitecore application.
In this part we will take a look at API to see what can be achieved using the search index.

To search existing index we need to get an index instance somehow. I'm not going to show the code examples that you would write with old search index. If you're intrested in it, check out this article Lecene Search Engine.
Additional layer of API for new search index resides under Sitecore.Search namespace. In order to get a search index object, you would need to use members of SearchManager class.
To get an index by name use this line of code:
Index indx = Search.Manager.GetIndex(index_name);
If you want to use default system index, you can simply call SystemIndex property of SearchManager class. In order to use Sitecore API to look up for some info in the index, you need to created a search context.
It's easy to do by calling CreateSearchContext method of the index object we got previously. It's also possible to create a search context by using one for the constructors of IndexSearchContext class. In this case it will be easy to run search queries by passing a search index instance as a parameter to the search context.
To search information, we need to create a query and run it in the search context. Sitecore API has a few classes that you can use to build search queries. Let's take a look at each of them:
FullTextQuery - this type of query searches the index by "_content" field. All information from text fields (such as "Single-Line Text", "Rich Text", "Multi-Line Text", "text", "rich text", "html", "memo") is stored there. Data in this field are indexed and tokenized. Which means that the search operations running on these data are very efficient.
FieldQuery - this type of query allows you to search any field that was added to the index. By default database crawler adds all item fields to the index.
CombinedQuery - this type of query was designed to allow you create complex queries with additional conditions. For instance to find items which have specific work in title and belong to some category. When you add search queries to this type of query, you need to supply QueryOccurance parameter. It's Enum type that has the following members:
- Must - it's a logical AND operator in Lucene boolean query.
- MustNot - it's a logical NOT operator in Lucene boolean query.
- Should - it's a logical OR operation in Lucene boolean query.
You can read more about this operators in Query Parser Syntax article.

All of these query types are derived from QueryBase class.
There is one thing left until we jump from the theory to some code samples. To run defined queries, you need to use one of Search methods of IndexSearchContext object.
Now let's create a couple of samples to see a real code that goes behind the theory.

Sample 1: Searching text fields.
// Next samples will skip lines with getting the index instance and creating the search context.
// Get an index object
Index indx = SearchManager.GetIndex("my_index");
// Create a search context
using (IndexSearchContext searchContext = indx.CreateSearchContext())
{
// In following examples I will be using QueryBase class to create search queries.
FullTextQuery ftQuery = new FullTextQuery("welcome");
SearchHits results = searchContext.Search(ftQuery);
}

Sample 2: Searching item fields
Let's say we want to find items classified by some category. There is a trick searching by GUIDs so let's say our category is just a string name.
.....
// FieldQuery ctor accepts two parambers. First is a field name. The other one is a value we're looking for.
QueryBase query = FieldQuery("category", "slr");
SearchHits results = searchContext.Search(query);
.....

Sample 3: Searching by multiple conditions
Turned out that your category parameter is not enough to get required results. You client is screaming that there are too many items and business users cannot find ones they are looking for (is there anything they can find?).
Obviously you have some additional fields that can help to find more strict results. Let's say there is a rating field with values from 1 to 5.
That's where CombinedQuery gets into the game.
.....
// CombinedQuery object has Add method that should be used to add search queries to it. That's why we cannot use base class variable here.
CombinedQuery query = new CombinedQuery();
QueryBase catQuery = new FieldQuery("category", "slr");
QueryBase ratQuery = new FieldQuery("rating", "5");
query.Add(catQuery, QueryOccurance.Must);
query.Add(ratQuery, QueryOccurance.Must);
var hits = searchContext.Search(query);
.....

All results are presented as SearchHits object. Now you should use of following methods of SearchHits object to get the results as Sitecore items:
- FetchResults(int, int) - returns search results as SearchResultCollection. First parameter is a start position of an item you want to start fetching results from. Second one is count of items you want to fetch. By calling this function as mentioned below, you can get all results at once:
var results = hits.FetchResults(0, hits.Length);

- Slice(int) - returns all results as IEnumerable collection.

- Slice(int, int) - this method has similar signature to FetchResults but returns results as IEnumerable collection.

Here are a couple of examples the way you can transform SearchHits object into Sitecore items.
Sample 4: using FetchResults
.....
SearchResultCollection results = hits.FetchResults(0, hits.Length);
IEnumerable searchItems = from hit in results
select hit.GetObject();
}
.....

Sample 5: using Slice
.....
IList searchItems = List();
foreach(var hit in hits.Slice(0))
{
ItemUri itemUri = new ItemUri(hit.Url);
if (itemUri != null)
{
Item item = ItemManager.GetItem(itemUri.ItemID, itemUri.Language, itemUri.Version, Factory.GetDatabase(itemUri.DatabaseName));
if (item != null)
{
searchItems.Add(item);
}
}
}
.....

It's worth to mention that some variations of Search method of IndexSearchContext class can accept Lucene.Net.Search.Query as a search query parameter. It becomes very useful when you need to create a complex query which cannot be built with Sitecore query types.

Searching GUIDs.

New search index has lots of useful built-in fields that help to build strict queries.
Besides standard fields it has the following fields that contain GUIDs in ShortID format:
- _links - contains all references to current item

- _path - contains ShortIDs for every parent item in the path relative to current item

- _template - contains GUID of item's tempalte.
NOTE: this field is supposed to have ShortID value instead of GUID one. This field should not be used in combined queries prior to Sitecore 6.2 releases.
If you decide to add custom fields to your search index and they should have GUID values, you need to store them as ShortID in lower case format. Otherwise search will not be able to find any results. The reason why it happens is because Lucene recognizes GUIDs and applies special parsing for them. It works fine if search query has only one field to look into. If it's combined/complex query then it fails to find anything even if it's correct.
So, remember if you need to filter search results by template, you will have to customize DatabaseCrawler to add another field (e.g. _shorttemplateid) to store item template id in ShortID format.

Sample 1: Find all item references

.....
QueryBase query = FieldQuery("_links", ShortID.Encode(item.ID).ToLowerInvariant());
SearchHits results = searchContext.Search(query);
.....

Sample 6: Find all items based on specified template

.....
// Prior to Sitecore 6.2 release, you will need to add and use _shorttemplateid field
QueryBase query = FieldQuery("_template", ShortID.Encode(item.ID).ToLowerInvariant());
SearchHits results = searchContext.Search(query);
.....

Sample 7: Find items that are descendants of a specified one
.....
QueryBase query = FieldQuery("_path", ShortID.Encode(item.ID).ToLowerInvariant());
SearchHits results = searchContext.Search(query);
.....

Sample 8: Find items of a parent and belong to a specific template
.....
CombinedQuery query = new CombinedQuery();
query.Add(new FieldQuery("_shorttemplateid", ShortID.Encode(templateId).ToLowerInvariant()), QueryOccurance.Must);
query.Add(new FieldQuery("_path", ShortID.Encode(parent.ID).ToLowerInvariant())), QueryOccurance.Must);
.....

That's all I wanted to tell about Lucene search index in Sitecore 6. I hope it will help Lucene beginners to better understand the concept and get up to speed with Lucene search index abilities.
Enjoy!

Monday, November 2, 2009

Working with Lucene Search Index in Sitecore 6. Part II - How it works

Here is second part of the Lucene search index overview for Sitecore 6. In this part we'll take a look at configuration settings and talk about how it works.

Sitecore 5 has Lucene engine as well. Let's step one Sitecore version back and see how Lucene works there. In web.config file there is a section /sitecore/indexes that contains Lucene index configuration. When index is configured, it should be added to /sitecore/databases/database/indexes section.
The web database does not have a search index by default. Even if you add it to aforementioned section, it won't work. Why? Because index configuration relies on HistoryEngine functionality. By default the web database does not have it. It's easy to add it though. Just add the HistoryEngine configuration section to the database.
You can find more configuration details from this article on SDN.
This index has the same configuration in Sitecore 6.
In addition to it, Sitecore 6 has a new Lucene index functionality. Which is more reliable and has Sitecore level API on top of Lucene one. In some cases you will still have to use Lucene API. For instance to create range queries.
Configuration settings for new search index located under /sitecore/search section.
The analyzer section defines a Lucene analyzer that is used to analyze and index content.
The categories section is used to categories search results. It's used for content tree search introduced in Sitecore 6. The search box is located right above the content tree in content editor.
The configuration section has indexes definitions with their configurations. An index definition should be created under /sitecore/search/configuration/indexes node.
First two parameters describe the index name and folder name where it should be stored:
<param desc="name">$(id)</param>
<param desc="folder">my_index_folderName</param>
Next setting is the analyzer that should be used for the index:
<Analyzer ref="search/analyzer" />
Lucene StandardAnalyzer covers most of the case scenarios. But it's possible to use any other analyzer if it's needed.
Following setting defines locations for the index:
<locations hint="list:AddCrawler">
It's possible to have multiple locations for one index. Moreover it's even possible to have content from different databases in the same index. Every child of the locations node has its own configuration for a particular part of the content. A name of location node is not predefined. You're welcome to name it the way you want. For example:
<locations hint="list:AddCrawler">
<sdn-site type="Sitecore.Search.Crawlers.DatabaseCrawler, Sitecore.Kernel">
...
</sdn-site>
</locations>
Every location has a database section. It defines indexing database for the location.
Then root section. The database crawler will index content beneath this path.
Next sibling node is the include section. Here it's possible to add templates items of which should be included to the index or excluded from it.
Example:
<include hint="list:IncludeTemplate">
<sampleItem>{76036F5E-CBCE-46D1-AF0A-4143F9B557AA}</sampleItem>
</include>
<include hint="list:ExcludeTemplate">
<layout>{3A45A723-64EE-4919-9D41-02FD40FD1466}</layout>
</include>
It does not make sense to use both of these settings for the one location. Use only one of them.
Next location setting is tags section. Here you can tag indexed content and use it during the search procedure.
Last setting is boost section. Here you have an ability to boost indexed content among other content that belongs to other locations.
And last but not the least, this search index uses the same HistoryEngine mechanism as old one. So, don't forget to copy configuration section from master database to a database where you want to add search index facilities to.

How it all works?
When an action performed on the item, database crawler updates entries in search index for the item. So that information in index is in sync with the one in database. How does it happen if "item:saved", "item:deleted", "item:renamed", "item:copied", "item:moved" do not have event handlers that trigger search index update? Thank to HistoryEngine that was mentioned several times already.
It is HistoryEngine that tracks any changes made to the item and fires appropriate event handler to process it.
IndexingManager is responsible for all operations to the search index. It subscribes to AddEntry event of HistoryEngine and as soon as an entry added to the History table, it triggers a job that updates the search index(es).
In web.config file there are a few settings that belong to indexing functionality.
  • Indexing.UpdateInterval - sets the interval between the IndexingManager checking its queue for pending actions. Default value is 5 minutes.
    What does it mean? If for whatever reason pending job was not executed, the IndexingManager will re-run it if it finds it in pending state after 5 minutes pass.
  • Indexing.UpdateJobThrottle - sets the minimum time to wait between individual index update jobs. Default value 1 second.
    When some operation is performed on the item, you can see this entry in Sitecore log file:
    INFO Starting update of index for the database 'databaseName' ( pending).
    This setting sets the interval between jobs like this. So that it does not overwhelm all CPU time if you're doing massive change to the items.
  • Indexing.ServerSpecificProperties - Indicates if server specific keys should be used for property values (such as 'last updated'). It's off by default.
    This setting is designed for content delivery environments in web farms. As web database is shared, there could be a situation when one server has updated its search indexes and changed History table in the database. Other servers won't update their indexes because HistoryEngine wouldn't indicate there was a change. This setting prevents situations like this.
Well... this is it for now. In next part we will take a look at Sitecore Lucene API and create some search queries with it.
Enjoy!