Coveo Live Monitoring not Indexing Items in Queue
We are using Coveo for Sitecore and occassionally, when I take a look at the queues in rabbitMQ, I'll notice a substantial amount of items(10,000-70,000 depending on the index) sitting in the queue despite live monitoring being turned on in the console for the associated index.
After looking into the problem I found an article from Coveo: Coveo for Sitecore troubleshooting that suggests turning Live Monitoring for the affected index off and then back on again in the admin tool. Although this tends to solve the problem in the short term (suddenly Coveo begins indexing the overflow of items) I was wondering why exactly this occurs and why I seem to find myself in the same situation a month or so later.
Is there a schedule I can implement in order to resolve this automatically? Or a means in which I can automate turning live monitoring off and then on again once a week or so in order to ensure it is working properly? Or, even more fundamental, is this an indication of a problem with the indexes?
The connection can fail is RabbitMQ crosses half the available amount of RAM on the server. This is shown as a high watermark in the Memory box of the RabbitMQ Management console: http://localhost:15672/
Usually, the service will return an error in the coveo index logs (CES7\Log) when this issue appears. The error would be something like: "RabbitMQ connection failed"
Keep an eye on the RAM consumption of the index and be sure to respect the recommended specs: https://onlinehelp.coveo.com/en/ces/7.0/administrator/coveoplatformhardwareandsoftware_requirements.htm
Additionally, we optimized the indexing process in September 2015 to reduce the memory usage of every items sent to the index. Upgrading might solve your issue for good.