Gravatar for

Question by AlexShapo, Feb 20, 2018 7:30 PM

How to customize source for specific URLs


I need to create a source which will contain only News & Media on a website.

The URL looks like this: and under that root we have all our news, which look like this:

How can I make Source crawl only links which looks like above? Currently it crwawling everything, even if I specify correct start link

1 Reply
Gravatar for

Answer by Etienne, Feb 20, 2018 8:44 PM

Check the "inclusion filters section" in this documentation page.

And tell me if it solves your question.

Gravatar for

Comment by AlexShapo, Feb 20, 2018 9:26 PM

Is the following correct way to setup filter? the Build Index fail with the message "Web no document indexed due to filters"

I also trying with regex, but failing because can't find documents:


Gravatar for

Comment by AlexShapo, Feb 22, 2018 2:23 PM

Looks like it does not work with URLs of multiple levels, the following works for me


Ask a question