No indexed document external source (web pages)
I added web pages type external source for a website we hosted in our infrastructure. We use on-premise Coveo and it sit in the same network as web servers. But it doesn't get any document indexed at all so far as I start building the index since yesterday. The coveo server is definitely has access to the web site as I can browse to it. I also check the log but no useful information at all.
I finally find the result as @Simon suggested. I start over from the beginning again and managed to find the option to not respecting the robot.txt in the advance section. (It just a single drop down list)
Here is another article mentioned about this robot.txt file. https://onlinehelp.coveo.com/en/cloud/addeditweb_source.htm
Thanks a lot @Simon