Gravatar for fcote@coveo.com

Question by fcote, Jul 18, 2016 11:08 AM

LINQ Predicate Statement Involving Sitecore fields and External Sources

When using predicate statements that involve Sitecore fields with the CoveoSearchContextOption code below, it seems to negate any external results coming back. This appears to be due to the external content not having the Sitecore fields but being put against the rules.

For example, using the code below, we still only want Sitecore content from the Sitecore content root and we want to make sure it has a presentation (layout).

// This creates some query filters
var predicate = Sitecore.ContentSearch.Linq.Utilities.PredicateBuilder.True<SearchResultItem>();
// Restrict Sitecore items to /sitecore/content and ensure they have a layout set
predicate = predicate.And(i => i.Path.Contains(contentRoot) && i["haspresentation"] == "1" && !i.ExcludeFromEndUserSearchResults);
// Free-text search for the searchTerm
predicate = predicate.And(i => i.Content == searchTerm);

I see the OR statement in the CES console that adds the external syscollection. It appears ​

Query
((@fz95xfullpath12345*="*sitecore/content*") (@fhaspresentation12345=="1") (NOT (@fez120xcludez32xfromz32xendz32xuserz32xsearchz32xresults12345==1)) (Kilts)
OR
(@syscollection="research_sites"))((@fz95xfullpath12345*="*sitecore/content*") (@fhaspresentation12345=="1") (NOT (@fez120xcludez32xfromz32xendz32xuserz32xsearchz32xresults38935==1)) (Kilts))
performed by ad\myUser [Sitecore Security Provider for DEV-intranet-dev.mySite.me]. 0 results in 0.109 seconds.

Similar to the use of coveoContext for ExternalCollections, should the predicate be supplied to the search context a different way?

Gravatar for josh.bair@gmail.com

Comment by jbairuc, Jul 19, 2016 7:16 PM

This is actually my issue…

Here is this the full code, with reference to the external collection (a crawled web site).

// This creates some query filters var predicate = Sitecore.ContentSearch.Linq.Utilities.PredicateBuilder.True<SearchResultItem>(); // Restrict Sitecore items to /sitecore/content and ensure they have a layout set predicate = predicate.And(i => i.Path.Contains(contentRoot) && i["haspresentation"] == "1" && !i.ExcludeFromEndUserSearchResults); // Free-text search for the searchTerm predicate = predicate.And(i => i.Content == searchTerm); // Create a Queryable to query the index IQueryable<SearchResultItem> queryable; // Create a Search Context // This will automatically add a filter expression for the source indexing this "index" content. using (var context = index.CreateSearchContext()) { // Cast the search context in a Coveo search context ICoveoSearchContext coveoContext = context as ICoveoSearchContext; // Add the external collection in the search context. It will be used in OR with the automatic source filter for the index coveoContext.SearchContextOptions = new CoveoSearchContextOptions { ExternalCollections = new List<string>() { "research_sites" }}; // Perform the query with the predicate filters defined above AND the automatic source filter from the search context. queryable = context.GetQueryable<ExtendedSearchResultItem>().Where(predicate); }

The issue is that the predicate part that restricts the Sitecore search ends up in the query on the side for the external collection as well.

Query: ((@fz95xfullpath12345="sitecore/content") (@fhaspresentation12345=="1") (NOT (@fez120xcludez32xfromz32xendz32xuserz32xsearchz32xresults12345==1)) (SearchTerm) OR (@syscollection="research_sites"))((@fz95xfullpath12345="sitecore/content") (@fhaspresentation12345=="1") (NOT (@fez120xcludez32xfromz32xendz32xuserz32xsearchz32xresults38935==1)) (SearchTerm))

Gravatar for josh.bair@gmail.com

Comment by jbairuc, Jul 20, 2016 7:37 PM

In my goal to get the external collection part alone, away from the Sitecore statements, I tried a RawQueryExpression:

// Cast the search context in a Coveo search context
                    ICoveoSearchContext coveoContext = context as ICoveoSearchContext;

                    // Add the external collection in the search context. It will be used in OR with the automatic source filter for the index 
                    //coveoContext.SearchContextOptions = new CoveoSearchContextOptions { ExternalCollections = new List<string>() { "research_sites" } };
                    coveoContext.SearchContextOptions = new CoveoSearchContextOptions
                    {
                        RawQueryExpression = " OR (@syscollection==\"research_sites\")" + "(" + GetQueryFromQueryString() + ")"
                    };

In the CES Console, the query looks good, but it's returning no results:

Query (@fz95xfullpath38935="sitecore/content*") (@fhaspresentation38935=="1") (NOT (@fez120xcludez32xfromz32xendz32xuserz32xsearchz32xresults38935==1)) (SearchTerm) OR (@syscollection=="research_sites")(SearchTerm) performed by [redacted] [Sitecore Security Provider for www.example.org]. 0 results in 0.14 seconds.

Gravatar for jflheureux@coveo.com

Comment by Jean-François L'Heureux, Jul 21, 2016 4:22 PM

What is the security on your "research_sites" collection indexed documents? Are they available for the "everyone" group of the "Sitecore Security Provider for www.example.org" security provider as explained in the documentation: https://developers.coveo.com/display/SitecoreV3/Displaying+External+Content+in+a+Search+Interface

Gravatar for jflheureux@coveo.com

Comment by Jean-François L'Heureux, Jul 21, 2016 4:23 PM

Which version of Coveo for Sitecore do you use (3.0 or 4.0, Cloud or on-premises configuration)?

Gravatar for josh.bair@gmail.com

Comment by jbairuc, Jul 22, 2016 9:42 AM

It's 3.0 - a version from early this year. I have followed that documentation and though the everyone group is in the collection permissions, not everyone is in it. As a result, other groups have had to be added. There are others who have run into this and it seems related to the security cache.

Gravatar for josh.bair@gmail.com

Comment by jbairuc, Jul 22, 2016 1:28 PM

Essentially I can call HTTP://Sitecore.example.org/Coveo/rest/?q=(syscollection==research_sites)SearchTerm, get external resulte back and see the Sitecore user it's using in the Coveo console.

It just happens to not work when the LINQ predicate contains the Sitecore-related conditions above.

1 Reply
Gravatar for josh.bair@gmail.com

Answer by jbairuc, Jul 26, 2016 3:37 PM

I added a redundant statement to the predicate to include the external collection (as well as keeping the ExternalCollections CoveoSearchContextOptions.

predicate = predicate.Or(i => i["syscollection"] == ("External Content"));

and

ICoveoSearchContext coveoContext = context as ICoveoSearchContext;
coveoContext.SearchContextOptions = new CoveoSearchContextOptions { ExternalCollections = new List<string>() { "External Content" } };

are both in place. This has somehow allowed it to work.

Console output:

Query ((((@fz95xfullpath38935*="*sitecore/content*") (@fhaspresentation38935=="1") (NOT (@fez120xcludez32xfromz32xendz32xuserz32xsearchz32xresults38935==1))) OR (@syscollection=="External Content") OR (@fez120xtension38935=="pdf") OR (@fez120xtension38935=="doc") OR (@fez120xtension38935=="docx") OR (@fez120xtension38935=="xls") OR (@fez120xtension38935=="xlsx") OR (@fez120xtension38935=="ppt") OR (@fez120xtension38935=="pptx")) (SearchTerm) ((NOT ((@fez120xtension38935=="pdf") OR (@fez120xtension38935=="doc") OR (@fez120xtension38935=="docx") OR (@fez120xtension38935=="xls") OR (@fez120xtension38935=="xlsx") OR (@fez120xtension38935=="ppt") OR (@fez120xtension38935=="pptx"))) OR (@fez120xtension38935=="pdf") OR (@fez120xtension38935=="doc") OR (@fez120xtension38935=="docx") OR (@fez120xtension38935=="xls") OR (@fez120xtension38935=="xlsx") OR (@fez120xtension38935=="ppt") OR (@fez120xtension38935=="pptx")) OR (@syscollection="External Content"))((((@fz95xfullpath38935*="*sitecore/content*") (@fhaspresentation38935=="1") (NOT (@fez120xcludez32xfromz32xendz32xuserz32xsearchz32xresults38935==1))) OR (@syscollection=="External Content") OR (@fez120xtension38935=="pdf") OR (@fez120xtension38935=="doc") OR (@fez120xtension38935=="docx") OR (@fez120xtension38935=="xls") OR (@fez120xtension38935=="xlsx") OR (@fez120xtension38935=="ppt") OR (@fez120xtension38935=="pptx")) (SearchTerm) ((NOT ((@fez120xtension38935=="pdf") OR (@fez120xtension38935=="doc") OR (@fez120xtension38935=="docx") OR (@fez120xtension38935=="xls") OR (@fez120xtension38935=="xlsx") OR (@fez120xtension38935=="ppt") OR (@fez120xtension38935=="pptx"))) OR (@fez120xtension38935=="pdf") OR (@fez120xtension38935=="doc") OR (@fez120xtension38935=="docx") OR (@fez120xtension38935=="xls") OR (@fez120xtension38935=="xlsx") OR (@fez120xtension38935=="ppt") OR (@fez120xtension38935=="pptx"))) performed by domain\user [Sitecore Security Provider for sitecore-web.example.org]. 276 results in 0.515 seconds.
Ask a question