How to customize source for specific URLs
I need to create a source which will contain only News & Media on a website.
The URL looks like this: http://www.site.com/news-and-media/releases and under that root we have all our news, which look like this:
How can I make Source crawl only links which looks like above? Currently it crwawling everything, even if I specify correct start link
Check the "inclusion filters section" in this documentation page.
And tell me if it solves your question.