ran into something similar before. i'd check include_tags="article" in call_extract() first, since in crawl mode links outside the article tag can disappear, and pages without it may come back empty and get marked as thin. once that's sorted the orphan label will actually mean smth, and counting unique referring pages instead of raw links would help too.
the md5 check marks every page in a duplicate group, the canonical one included, so picking one url per group as the target is worth doing. wrapping the map and crawl calls in error handling and logging robots.txt blocked urls would keep the inventory honest about what it covers.
OnlineProxy
ran into something similar before. i'd check include_tags="article" in call_extract() first, since in crawl mode links outside the article tag can disappear, and pages without it may come back empty and get marked as thin. once that's sorted the orphan label will actually mean smth, and counting unique referring pages instead of raw links would help too.
the md5 check marks every page in a duplicate group, the canonical one included, so picking one url per group as the target is worth doing. wrapping the map and crawl calls in error handling and logging robots.txt blocked urls would keep the inventory honest about what it covers.