Repository navigation
Add do_not_retry_timeouts and use it in push_items() #1133
Description
DatasetClient.push_items() and KeyValueStoreClient.set_record() go through the normal retry loop (dataset.py#L490-L516, key_value_store.py#L357-L404). A timeout is a retryable transport error, so a push whose response arrives after the timeout is sent again. The API may already have stored the first batch, and the dataset then holds the items twice. For set_record() the retry overwrites the same key, so a duplicate is less of a problem there, but a large upload is still sent again in full.
The JS client has a doNotRetryTimeouts request option that stops retrying on a timeout (base.ts#L133-L134, #L636-L640). pushItems() always sets it (dataset.ts#L400-L409) and setRecord() exposes it as an option (key_value_store.ts#L76-L79, #L586). Python has no equivalent. #990 covered retried non-idempotent requests in general and was closed in favor of a platform-side fix (apify/apify-core#30485, closed as a duplicate of apify/apify-core#7479, Idempotent API), which has not landed yet. This asks only for the client-side switch.
Expected behavior
HttpClient.call() accepts do_not_retry_timeouts. push_items() sets it, and set_record() takes it as an optional keyword argument, sync and async.
✍️ Drafted by Claude Code
Activity
vdusek commentedon Oct 16, 2026
Closing as not planned (at least for now). This is a client-side workaround for #990, and it would add a public parameter that we'd want to remove again once the platform fixes the underlying problem. Duplicate items from retried pushes are also pretty rare. If there's demand, we can add it the same way the JS client has it. However, the proper fix is the idempotent API - apify/apify-core#7479.