Need the ability to do de-duplication conversion prior to receiving a query
Need the ability to do more robust deduplication
Protocols and extensions on URLs are not easily deduplicated. Such as…HTTP vs HTTPs, and treating trailing slash â/â, file and extensions (.htm, .html, aspx) as synonyms to deduplicate from
Current deduplication efforts have cause other issues such as incorrect pagination and result counts for the end user.