The 20 item limit is definitely a pain but that raise for status call will instantly kill your whole crawl on a single server error. I would add some exponential backoff and track schema drift with JSON diffs to catch field changes before they break everything. Deduplicating is also a huge headache since the unique job id is only exposed in the detail endpoint. You end up having to do expensive extra fetches for every single job just to get accurate data. This completely changes the cost model so you really have to calculate those API hits upfront before trying to run this across hundreds of companies