Web Pages connector source not retrieving all pages
We have an external source using the Web connector for our Brightcove videos. Right now the Site URL is pointing to several URLs, including:
This page has thousands of Brightcove videos, but when I Save and Rebuild the source, I am only getting 37 results. When the rebuild completes, it reports 217 items were processed but only 37 items shown in the Content Browser. My crawl depth is 5000 pages, which should be fine because there are only around 3000 videos total. I don't think I've changed any other settings in the source.
I'm not really sure what the problem was; but, with Coveo support's help, we found a sitemap on the site that we could use with the Sitemap Connector. We created a new Sitemap source and pointed it to the sitemap. Now all videos are being pulled in.