the gzip check goes by the URL ending and content type, so a sitemap.xml.gz?ts=123 with a generic type might slip through, and checking data.startswith(b"\x1f\x8b") would cover that. root.iter() can also pick up image loc entries, so findall with the unused NS map plus the current loop as a fallback could keep things clean.
for robots.txt, cutting anything after a hash and stripping the line first would handle trailing comments and indented Sitemap lines. might also be worth a size cap in fetch, since one big compressed sitemap gets fully decompressed into memory