Gravatar for

Question by Cris Corra, Jul 1, 2014 11:01 AM

Is there a way to throttle the indexing of a public website?

One of our sources is an external website that hosts online journals. The owners of the site are requesting we limit the spidering of their site to one request per second. Is this possible?

-Cris Corra

1 Reply
Gravatar for

Answer by Luc Bergeron, Jul 2, 2014 10:55 AM

Yes, you are looking for the "Delay between downloads" setting of your CES source.

You will find more information on this page:

Hope this helps

Gravatar for

Comment by Cris Corra, Jul 3, 2014 8:38 AM

Thank you! This is exactly what we were looking for.

Ask a question