Gravatar for lukas.kozak@bwinparty.com

Question by lukaskozak, Apr 2, 2015 3:12 AM

Sitecore indexing performance

I'm trying to use the false switch to exclude Sitecore fields from Coveo to improve indexing performance. However, this setting doesn't have any effect on the number of fields in Coveo. I'm not going to need to use the LINQ layer to query those fields directly, so why index them?

However I want to be able to perform a full-text query, so there should be a 'big' field containing text from all the fields in Sitecore, similar to the '_content' field which is used in Sitecore-SOLR integration.

How can I achieve that? Is this the right approach in Coveo? How many items per minute should the Coveo plugin in Sitecore index on average?

1 Reply
Gravatar for vseguin@coveo.com

Answer by Vincent Séguin, Apr 2, 2015 7:33 AM

Hi,

Indexing performance is not affected by the number of fields. Indexing performance will be affected by how "big" your items are. For instance, pdfs and media librarys item will be slower to index than regular items.

If you want to perform a full-text query, you can either configure some specific fields to be free-text searchable in the Coveo.SearchProvider.config (see https://developers.coveo.com/display/public/SC201503/Understanding+the+Coveo+Search+Provider%27s+Configuration+File) or also ensure that the data you want is in the body of your document. To achieve that, you can develop your own PostItemProcessingPipeline processor and put anything in the "BinaryData" property of the document your index.

See more here : https://developers.coveo.com/display/public/SC201503/Understanding+the+Indexing+and+Search+Pipelines and https://developers.coveo.com/display/public/SC201503/Indexing+Documents+with+Basic+HTML+Content https://developers.coveo.com/display/public/SC201503/Indexing+Documents+with+HTML+Content+Processor

(The lattest two will set the HTML representation of your item in the body).

Gravatar for lukas.kozak@bwinparty.com

Comment by lukaskozak, Apr 9, 2015 3:50 AM

What is the switch 'indexallfields' meant for then?

Gravatar for vseguin@coveo.com

Comment by Vincent Séguin, Apr 9, 2015 7:18 AM

It will set some Coveo Metadata on the document, but will not index the Sitecore fields. Caution : fields will still be created, but will not be populated on documents.

Gravatar for praful.memane@towerswatson.com

Comment by prafulmemane, Apr 22, 2015 6:34 PM

Is there an easy configuration way to set indexallfields false and add a list of fields per template to be indexed?

Gravatar for vseguin@coveo.com

Comment by Vincent Séguin, Apr 22, 2015 6:52 PM

What would be the use case?

Gravatar for pavan.omtri@towerswatson.com

Comment by psomtri, Apr 26, 2015 12:14 AM

Wanted to comment on the statement "Indexing performance will be affected by how "big" your items are". Even if the items are not media items but has many number of fields, especially RTFs with many lines of content, Wouldn't that be considered a big item ? Wouldn't such items slow down indexing? In that case wouldn't we want to exclude some fields from being indexed? I think this would be valid use case for this question.

Also I would like to know what is the purpose of and how is it supposed to be used.

Gravatar for vseguin@coveo.com

Comment by Vincent Séguin, Apr 27, 2015 7:11 AM

The includeFIelds are valid for Sitecore's default search and are not processed by Coveo. Like i said, the fields are not really impacting the size of an item in the index. An item with much more complexe binary data will slow down the indexing, because it needs to be converted. The fields value are not being converted, they are indexed as raw.

You could put this field in a tag if you really need it to be excluded.

Gravatar for praful.memane@towerswatson.com

Comment by prafulmemane, Apr 30, 2015 11:11 AM

Vincent, Sorry, I missed your comment. I want to exclude all fields from being indexed and include only few fields to be indexed. So in ideal scenario false …… {8CDC337E-A112-42FB-BBB4-4143751E123F} {7RFC337E-A982-42FB-BCB4-4143751TY3F} This would only index Title and Body Fields.

Following this article here - https://developers.coveo.com/display/public/SC201502/Customizing+the+Indexing+Parameters

Gravatar for praful.memane@towerswatson.com

Comment by prafulmemane, May 1, 2015 11:52 AM

Can someone answer this question please?

Gravatar for vseguin@coveo.com

Comment by Vincent Séguin, May 1, 2015 11:53 AM

I'm still not sure why you would exclude all fields from being indexed…. but the example you have should work.

Ask a question