Gravatar for loebrandy@gmail.com

Question by rloeb, Jul 13, 2016 4:04 PM

Sitecore core database bucketed item indexing fail.

We are encountering an issue after having upgraded to Sitecore 8. We are indexing the core database, and have some bucketed items. They are not appearing in the index. Other items, such as the bucket folders and media items are being indexed.

The items in question are profile items derived from the user template.

We see the following error in the logs:

ManagedPoolThread #6 14:45:10 INFO  Adding 8 to the field set.
ManagedPoolThread #6 14:45:10 ERROR An error occurred while adding the field syssprelativelisturi : An exception occurred while trying to add the field syssprelativelisturi in the field set Field set for Coveo_core_index - ROMAN-HP-taxsearch-dev : class MerlinConfig::DuplicateKeyException: There is a duplicate key sequence syssprelativelisturi.
ManagedPoolThread #6 14:45:10 ERROR An error occurred while adding the field syssplistname : An exception occurred while trying to add the field syssplistname in the field set Field set for Coveo_core_index - ROMAN-HP-taxsearch-dev : class MerlinConfig::DuplicateKeyException: There is a duplicate key sequence syssplistname.
ManagedPoolThread #6 14:45:10 ERROR An error occurred while adding the field sysclickableuri : An exception occurred while trying to add the field sysclickableuri in the field set Field set for Coveo_core_index - ROMAN-HP-taxsearch-dev : class MerlinConfig::DuplicateKeyException: There is a duplicate key sequence sysclickableuri.
ManagedPoolThread #6 14:45:10 ERROR An error occurred while adding the field sysrowid : An exception occurred while trying to add the field sysrowid in the field set Field set for Coveo_core_index - ROMAN-HP-taxsearch-dev : class MerlinConfig::DuplicateKeyException: There is a duplicate key sequence sysrowid.
ManagedPoolThread #6 14:45:10 ERROR An error occurred while adding the field sysfilename : An exception occurred while trying to add the field sysfilename in the field set Field set for Coveo_core_index - ROMAN-HP-taxsearch-dev : class MerlinConfig::DuplicateKeyException: There is a duplicate key sequence sysfilename.
ManagedPoolThread #6 14:45:10 ERROR An error occurred while adding the field sysfiletype : An exception occurred while trying to add the field sysfiletype in the field set Field set for Coveo_core_index - ROMAN-HP-taxsearch-dev : class MerlinConfig::DuplicateKeyException: There is a duplicate key sequence sysfiletype.
ManagedPoolThread #6 14:45:10 ERROR An error occurred while adding the field syssource : An exception occurred while trying to add the field syssource in the field set Field set for Coveo_core_index - ROMAN-HP-taxsearch-dev : class MerlinConfig::DuplicateKeyException: There is a duplicate key sequence syssource.
ManagedPoolThread #6 14:45:10 ERROR An error occurred while adding the field systitle : An exception occurred while trying to add the field systitle in the field set Field set for Coveo_core_index - ROMAN-HP-taxsearch-dev : class MerlinConfig::DuplicateKeyException: There is a duplicate key sequence systitle.
ManagedPoolThread #6 14:45:10 INFO  Using existing source 'Coveo_core_index - ROMAN-HP-taxsearch-dev'.

…which may or may not be related. Any ideas? Thanks.

Gravatar for jflheureux@coveo.com

Comment by Jean-François L'Heureux, Jul 13, 2016 4:27 PM

Hi,

Can you give me your build numbers of:

  • Sitecore 8.0.???
  • Coveo for Sitecore 3.0.???? or 4.0.???
  • Coveo Enterprise Search 7.0.????, if you are using it.

Thank you

Gravatar for jflheureux@coveo.com

Comment by Jean-François L'Heureux, Jul 13, 2016 4:30 PM

Can you also give me the content of your Coveo default index configuration section please? I think you have fields configuration that doesn't belong there.

Gravatar for loebrandy@gmail.com

Comment by rloeb, Jul 13, 2016 4:43 PM

Current Coveo for Sitecore version: 3.0.1312.0

Current Sitecore version: 8.1.160519

Current Coveo Enterprise Search version: 7.0 x64 Build 8388.0

We actually have many more than this…but the issue is we wanted to mark some sys fields as isExternal=true because we are using them in queries accros sources and don't want hash added by .net library that resolves field names. Similar to isFacet, isSortable. Didn't seem to cause problems until recent upgrade to Coveo & Sitecore (7->8).

Let me know if you really need all fieldNames, and if there is more appropriate way for us to change properties on the sys fields.

Thanks.

<fieldNames hint="raw:AddFieldByFieldName">
          <fieldType fieldName="syssprelativelisturi" isExternal="true" isFacet="true" isSortable="true" includeForFreeTextSearch="true" isMultiValue="false" settingType="Coveo.Framework.Configuration.FieldConfiguration, Coveo.Framework" type="Sitecore.Data.ID, Sitecore.Kernel" returnType="System.String" />
          <fieldType fieldName="syssplistname" isExternal="true" isFacet="true" isSortable="true" includeForFreeTextSearch="true" isMultiValue="false" settingType="Coveo.Framework.Configuration.FieldConfiguration, Coveo.Framework" type="Sitecore.Data.ID, Sitecore.Kernel" returnType="System.String" />
          <fieldType fieldName="sysclickableuri" isExternal="true" isFacet="true" isSortable="true" includeForFreeTextSearch="true" isMultiValue="false" settingType="Coveo.Framework.Configuration.FieldConfiguration, Coveo.Framework" type="Sitecore.Data.ID, Sitecore.Kernel" returnType="System.String" />
          <fieldType fieldName="urllink" isExternal="true" isFacet="true" isSortable="true" includeForFreeTextSearch="true" isMultiValue="false" settingType="Coveo.Framework.Configuration.FieldConfiguration, Coveo.Framework" type="Sitecore.Data.ID, Sitecore.Kernel" returnType="System.String" />
          <fieldType fieldName="sysrowid" isExternal="true" isFacet="true" isSortable="true" includeForFreeTextSearch="true" isMultiValue="false" settingType="Coveo.Framework.Configuration.FieldConfiguration, Coveo.Framework" type="Sitecore.Data.ID, Sitecore.Kernel" returnType="System.String" />
          <fieldType fieldName="sysfilename" isExternal="true" isFacet="true" isSortable="true" includeForFreeTextSearch="true" isMultiValue="false" settingType="Coveo.Framework.Configuration.FieldConfiguration, Coveo.Framework" type="Sitecore.Data.ID, Sitecore.Kernel" returnType="System.String" />
          <fieldType fieldName="sysfiletype" isExternal="true" isFacet="true" isSortable="true" includeForFreeTextSearch="true" isMultiValue="false" settingType="Coveo.Framework.Configuration.FieldConfiguration, Coveo.Framework" type="Sitecore.Data.ID, Sitecore.Kernel" returnType="System.String" />
          <fieldType fieldName="syssource" isExternal="true" isFacet="true" isSortable="true" includeForFreeTextSearch="true" isMultiValue="false" settingType="Coveo.Framework.Configuration.FieldConfiguration, Coveo.Framework" type="Sitecore.Data.ID, Sitecore.Kernel" returnType="System.String" />
          <fieldType fieldName="systitle" isExternal="true" isFacet="true" isSortable="true" includeForFreeTextSearch="true" isMultiValue="false" settingType="Coveo.Framework.Configuration.FieldConfiguration, Coveo.Framework" type="Sitecore.Data.ID, Sitecore.Kernel" returnType="System.String" />
          <fieldType fieldName="title" isExternal="true" isFacet="true" isSortable="true" includeForFreeTextSearch="true" isMultiValue="false" settingType="Coveo.Framework.Configuration.FieldConfiguration, Coveo.Framework" type="Sitecore.Data.ID, Sitecore.Kernel" returnType="System.String" />
Gravatar for loebrandy@gmail.com

Comment by rloeb, Jul 13, 2016 5:11 PM

Is there a problem with us naming this index Covecoreindex? Seems like maybe under 8 this is not a good idea?

Gravatar for loebrandy@gmail.com

Comment by rloeb, Jul 13, 2016 5:11 PM

Coveocoreindex

Gravatar for jflheureux@coveo.com

Comment by Jean-François L'Heureux, Jul 13, 2016 5:13 PM

Defining non-Sitecore fields in the raw:AddFieldByFieldNameconfiguration section is not the proper way to define them as external fields.

There is a separate configuration section exactly for this purpose. It is named <externalFields hint="raw:AddExternalField"> and is located in the Coveo.SearchProvider.config file. Here's the default content of this element:

<!-- Fields in this section are fields that doesn't exist on Sitecore items, but exist in CES or in external sources. -->
<externalFields hint="raw:AddExternalField">
  <field fieldName="sysauthor" />
  <field fieldName="syscollection" />
  <field fieldName="sysconcepts" />
  <field fieldName="sysdate" />
  <field fieldName="syssource" />
  <field fieldName="sysrowid" />
</externalFields>

You can add the other Coveo system field in this list to have them treated as external fields. This way, Coveo for Sitecore won't try to create those fields in the CES field sets.

To change their options (facet, sortable, free-text search…), you need to do the changes directly in Coveo Enterprise Search Administration Tool as they are non-Sitecore fields. You don't need to set the options in the Coveo for Sitecore configuration for external fields.

More information: https://developers.coveo.com/display/SitecoreV3/Understanding+the+Coveo+Search+Provider's+Configuration+File#UnderstandingtheCoveoSearchProvider'sConfigurationFile-externalFields

That being said, it probably won't fix your bucket items problem but you will have a lot less errors in your log files and will be able to better spot errors related to the bucket items.

1 Reply
Gravatar for loebrandy@gmail.com

Answer by rloeb, Jul 13, 2016 8:37 PM

Never mind…we resolved…seems like totally unrelated problem. Sorry.

Gravatar for jflheureux@coveo.com

Comment by Jean-François L'Heureux, Jul 13, 2016 8:55 PM

Do you have the solution for other users who might have the same problem in the future?

Ask a question