Gravatar for ebnoble32@charter.net

Question by sirfergall, Oct 16, 2015 3:01 PM

Coveo Live Monitoring not Indexing Items in Queue

We are using Coveo for Sitecore and occassionally, when I take a look at the queues in rabbitMQ, I'll notice a substantial amount of items(10,000-70,000 depending on the index) sitting in the queue despite live monitoring being turned on in the console for the associated index.

After looking into the problem I found an article from Coveo: Coveo for Sitecore troubleshooting that suggests turning Live Monitoring for the affected index off and then back on again in the admin tool. Although this tends to solve the problem in the short term (suddenly Coveo begins indexing the overflow of items) I was wondering why exactly this occurs and why I seem to find myself in the same situation a month or so later.

Is there a schedule I can implement in order to resolve this automatically? Or a means in which I can automate turning live monitoring off and then on again once a week or so in order to ensure it is working properly? Or, even more fundamental, is this an indication of a problem with the indexes?

1 Reply
Gravatar for slangevin@coveo.com

Answer by Simon, Oct 16, 2015 4:23 PM

Hi,

The connection can fail is RabbitMQ crosses half the available amount of RAM on the server. This is shown as a high watermark in the Memory box of the RabbitMQ Management console: http://localhost:15672/

Usually, the service will return an error in the coveo index logs (CES7\Log) when this issue appears. The error would be something like: "RabbitMQ connection failed"

Keep an eye on the RAM consumption of the index and be sure to respect the recommended specs: https://onlinehelp.coveo.com/en/ces/7.0/administrator/coveoplatformhardwareandsoftware_requirements.htm

Additionally, we optimized the indexing process in September 2015 to reduce the memory usage of every items sent to the index. Upgrading might solve your issue for good.

Regards,
Simon

Gravatar for ebnoble32@charter.net

Comment by sirfergall, Oct 19, 2015 11:44 AM

Thanks Simon.

I'll have to take a look at upgrading to the September release.

I did find a few lines in the System logs that indicated an issue connecting to the RabbitMQ:

System2015-10-14.log:4029:212311656238069888|ERROR|2015-10-14|20:05:15|0|Unable to start the crawler "CESCC.DotNetCrawler#Queue": class Merlin::CustomCrawlersException: The custom crawler "CESCC.DotNetCrawler#Queue" produced the following error: Unable to reach RabbitMQ : None of the specified endpoints were reachable||| System2015-10-14.log:4045:212311656238070033|FATAL|2015-10-14|20:07:40|0|Unhandled exception in KIEC: TransactionThread - class Merlin::CustomCrawlersException: The custom crawler "CESCC.DotNetCrawler#Queue" produced the following error: Unable to reach RabbitMQ : None of the specified endpoints were reachable|||

Is this the error you're referring to?

Gravatar for slangevin@coveo.com

Comment by Simon, Oct 19, 2015 1:25 PM

Yes, this is exactly it

Gravatar for ebnoble32@charter.net

Comment by sirfergall, Oct 19, 2015 1:59 PM

Okay. Well that gives me some things to look into. I'll mark your answer as accepted since adding more resources or upgrading to the September release isn't something that will happen in a day :)

Thanks!

Ask a question