fix(sdk): align Python response and retry semantics with Anthropic (#8)

Author: moonyue-w

CHANGELOG.md

@@ -5,8 +5,16 @@ existing `0.1.0` release; earlier development prereleases are not listed.
 
 ## [Unreleased]
 
+### Added
+
+- Export `RequestTooLargeError` for HTTP 413 and `OverloadedError` for HTTP 529 from `qca` and `qca.common`, matching Anthropic's Python SDK.
+- Add `_strict_response_validation=True` to all four clients to require Pydantic validation of responses, including pagination and SSE. The setting is retained by `with_options` and response views.
+
 ### Changed
 
+- **Breaking:** Responses now use lenient model construction by default, matching Anthropic's Python SDK. Unexpected field types are preserved instead of raising `APIResponseValidationError`; nested models, extra fields, and request IDs remain available. Enable `_strict_response_validation=True` to retain the previous schema validation behavior. Invalid JSON and invalid download URLs still raise in either mode.
+- **Breaking:** HTTP 529 now raises `OverloadedError`, which inherits directly from `APIStatusError`, rather than `InternalServerError`. Callers catching `InternalServerError` for overloads should also catch `OverloadedError` or use `APIStatusError`.
+- Honor positive `Retry-After` and `Retry-After-Ms` delays above 60 seconds, capped at 4,294,967 seconds. Zero and negative delays use exponential backoff; numeric millisecond headers take precedence. Retry eligibility is unchanged.
 - Forward and Managed clients, both synchronous and asynchronous, now default to a 5-second connection timeout and 600-second read, write, and connection-pool timeouts, matching Anthropic's Python SDK. SSE streams can continue beyond 10 minutes while data keeps arriving within the read timeout.
 
 ## [0.1.0]

CONTRIBUTING.md

@@ -34,6 +34,16 @@ The default test command excludes account-backed integration tests and must not
 
 Never commit `.env.live`, tokens, credentials, generated logs, or test output. Integration scenarios must register cleanup immediately after creating a resource. Run them explicitly with `QODER_RUN_LIVE=1`; they are not part of public pull-request CI.
 
+The Forward and Managed integration files also include six `strict_response_contract` checks using a separate client with `_strict_response_validation=True`: model, template/agent, and session lists. They send only GET requests, inspect the first page with `limit=1` where supported, and allow empty lists. An empty list checks the response envelope; item schemas are checked when items exist. Returned items must have a non-empty string ID, and `data` must be present even when empty. Existing business scenarios continue to use the default lenient response parsing.
+
+To run only these read-only checks:
+
+```bash
+QODER_RUN_LIVE=1 QODER_LIVE_ENV_FILE=.env.live uv run pytest tests/integration -m integration -k strict_response_contract -v
+```
+
+These checks are included in the existing `test-live`, `test-live-managed`, and `test-live-all` targets. Offline regression tests use the same strict-client fixture and assertions with a mock HTTP transport to verify schema failures without credentials or network access.
+
 ## API and contract changes
 
 When adding or changing an endpoint:

README.md

@@ -203,7 +203,7 @@ Events are also readable after the fact through `client.sessions.events.list(ses
 
 ## Handling errors
 
-`APIConnectionError` is raised when the request never reached the API; `APITimeoutError` is its timeout subclass. A non-2xx status raises an `APIStatusError` subclass, and a response that cannot be decoded into its declared type raises `APIResponseValidationError`. All of them derive from `qca.APIError`.
+`APIConnectionError` is raised when the request never reached the API; `APITimeoutError` is its timeout subclass. A non-2xx status raises an `APIStatusError` subclass. Invalid JSON, invalid download URLs, and response schema mismatches when strict response validation is enabled raise `APIResponseValidationError`. All of them derive from `qca.APIError`.
 
 ```python
 from qca import APIConnectionError, APIStatusError, APITimeoutError
@@ -225,13 +225,29 @@ except APIStatusError as exc:
 | 403 | `PermissionDeniedError` |
 | 404 | `NotFoundError` |
 | 409 | `ConflictError` |
+| 413 | `RequestTooLargeError` |
 | 422 | `UnprocessableEntityError` |
 | 429 | `RateLimitError` |
-| 5xx | `InternalServerError` |
+| 529 | `OverloadedError` |
+| other 5xx | `InternalServerError` |
 | other | `APIStatusError` |
 
 `code` and `type` are read from the error body and are `None` when the server omits them, so log `message` and `request_id` as well. A non-JSON error body is kept verbatim in `.body`.
 
+`RequestTooLargeError` and `OverloadedError` inherit directly from `APIStatusError`, matching Anthropic. Code that previously caught `InternalServerError` for status 529 should now catch `OverloadedError` as well, or catch `APIStatusError` for all HTTP errors.
+
+## Response validation
+
+By default, responses are constructed into models without requiring every field to match the declared schema, matching Anthropic's Python SDK. Nested objects are still converted to models and unknown fields are retained. Unexpected field values may keep their original type, so type annotations describe the expected API schema rather than guaranteeing the runtime value.
+
+To require Pydantic response validation and raise `APIResponseValidationError` on a schema mismatch, enable the constructor option:
+
+```python
+client = Forward(_strict_response_validation=True)
+```
+
+The option is available on `Forward`, `Managed`, `AsyncForward`, and `AsyncManaged`, and is preserved by `with_options`, raw and streaming response views, pagination, and SSE parsing. Invalid JSON and invalid download URLs still raise `APIResponseValidationError` in either mode. Direct construction or validation of model classes continues to use Pydantic validation.
+
 ## Request IDs
 
 Every response model carries the `x-request-id` of the call that produced it, and errors expose the same value. Include it when reporting a problem.
@@ -245,6 +261,8 @@ print(identity._request_id)
 
 Certain errors are retried twice by default with exponential backoff. GET and HEAD requests, and any request carrying an idempotency key, are retried on connection errors, 408, 429, and 5xx; other requests are retried on 429 only. A 409 is never retried automatically, and an SSE stream that has already been established is never retried. Within those rules the SDK honors `x-should-retry` and a valid `Retry-After-Ms` or `Retry-After`.
 
+Positive server-requested delays, including values over 60 seconds, are honored up to 4,294,967 seconds, matching Anthropic's Python SDK. A numeric `Retry-After-Ms` takes precedence over `Retry-After`, which accepts seconds or an HTTP date. Zero, negative, or invalid delays fall back to exponential backoff. A server-requested wait can therefore exceed the timeout configured for an individual HTTP attempt.
+
 ```python
 client = Forward(max_retries=0)  # disable for all requests
 client.with_options(max_retries=5).sessions.list()  # or override per call site

docs/api/reference.md

@@ -66,7 +66,8 @@ def __init__(*,
              default_headers: Mapping[str, str] | None = None,
              default_query: Mapping[str, Any] | None = None,
              http_client: httpx.Client | None = None,
-             credential: Credential | None = None) -> None
+             credential: Credential | None = None,
+             _strict_response_validation: bool = False) -> None
 ```
 
 [[view_source]](https://github.com/QoderAI/qoder-cloud-agents-sdk-python/blob/main/src/qca/common/_base_client.py)
@@ -143,7 +144,8 @@ def __init__(*,
              default_headers: Mapping[str, str] | None = None,
              default_query: Mapping[str, Any] | None = None,
              http_client: httpx.AsyncClient | None = None,
-             credential: Credential | AsyncCredential | None = None) -> None
+             credential: Credential | AsyncCredential | None = None,
+             _strict_response_validation: bool = False) -> None
 ```
 
 [[view_source]](https://github.com/QoderAI/qoder-cloud-agents-sdk-python/blob/main/src/qca/common/_base_client.py)
@@ -356,6 +358,16 @@ class ConflictError(APIStatusError)
 
 [[view_source]](https://github.com/QoderAI/qoder-cloud-agents-sdk-python/blob/main/src/qca/common/_exceptions.py)
 
+<a id="qca.common._exceptions.RequestTooLargeError"></a>
+
+## RequestTooLargeError
+
+```python
+class RequestTooLargeError(APIStatusError)
+```
+
+[[view_source]](https://github.com/QoderAI/qoder-cloud-agents-sdk-python/blob/main/src/qca/common/_exceptions.py)
+
 <a id="qca.common._exceptions.UnprocessableEntityError"></a>
 
 ## UnprocessableEntityError
@@ -386,6 +398,16 @@ class InternalServerError(APIStatusError)
 
 [[view_source]](https://github.com/QoderAI/qoder-cloud-agents-sdk-python/blob/main/src/qca/common/_exceptions.py)
 
+<a id="qca.common._exceptions.OverloadedError"></a>
+
+## OverloadedError
+
+```python
+class OverloadedError(APIStatusError)
+```
+
+[[view_source]](https://github.com/QoderAI/qoder-cloud-agents-sdk-python/blob/main/src/qca/common/_exceptions.py)
+
 <a id="qca.common._exceptions.status_error"></a>
 
 #### status\_error
@@ -470,7 +492,11 @@ def to_json(*,
 #### parse\_response
 
 ```python
-def parse_response(cast_to: Any, data: Any, response: Any) -> Any
+def parse_response(cast_to: Any,
+                   data: Any,
+                   response: Any,
+                   *,
+                   strict: bool = False) -> Any
 ```
 
 [[view_source]](https://github.com/QoderAI/qoder-cloud-agents-sdk-python/blob/main/src/qca/common/_models.py)
@@ -1314,7 +1340,10 @@ class BaseStream(Generic[T])
 #### \_\_init\_\_
 
 ```python
-def __init__(response: httpx.Response, cast_to: Any) -> None
+def __init__(response: httpx.Response,
+             cast_to: Any,
+             *,
+             strict: bool = False) -> None
 ```
 
 [[view_source]](https://github.com/QoderAI/qoder-cloud-agents-sdk-python/blob/main/src/qca/common/_streaming.py)
@@ -1345,7 +1374,10 @@ class Stream(BaseStream[T], Iterator[T])
 #### \_\_init\_\_
 
 ```python
-def __init__(response: httpx.Response, cast_to: Any) -> None
+def __init__(response: httpx.Response,
+             cast_to: Any,
+             *,
+             strict: bool = False) -> None
 ```
 
 [[view_source]](https://github.com/QoderAI/qoder-cloud-agents-sdk-python/blob/main/src/qca/common/_streaming.py)
@@ -1405,7 +1437,10 @@ class AsyncStream(BaseStream[T], AsyncIterator[T])
 #### \_\_init\_\_
 
 ```python
-def __init__(response: httpx.Response, cast_to: Any) -> None
+def __init__(response: httpx.Response,
+             cast_to: Any,
+             *,
+             strict: bool = False) -> None
 ```
 
 [[view_source]](https://github.com/QoderAI/qoder-cloud-agents-sdk-python/blob/main/src/qca/common/_streaming.py)

src/qca/__init__.py

@@ -35,12 +35,18 @@
 from .common import (
     NotGiven as NotGiven,
 )
+from .common import (
+    OverloadedError as OverloadedError,
+)
 from .common import (
     PermissionDeniedError as PermissionDeniedError,
 )
 from .common import (
     RateLimitError as RateLimitError,
 )
+from .common import (
+    RequestTooLargeError as RequestTooLargeError,
+)
 from .common import (
     UnprocessableEntityError as UnprocessableEntityError,
 )

src/qca/common/__init__.py

@@ -28,12 +28,18 @@
 from ._exceptions import (
     NotFoundError as NotFoundError,
 )
+from ._exceptions import (
+    OverloadedError as OverloadedError,
+)
 from ._exceptions import (
     PermissionDeniedError as PermissionDeniedError,
 )
 from ._exceptions import (
     RateLimitError as RateLimitError,
 )
+from ._exceptions import (
+    RequestTooLargeError as RequestTooLargeError,
+)
 from ._exceptions import (
     UnprocessableEntityError as UnprocessableEntityError,
 )

src/qca/common/_base_client.py

@@ -4,7 +4,7 @@
 import os
 import random
 import time
-from email.utils import parsedate_to_datetime
+from email.utils import mktime_tz, parsedate_tz
 from typing import Any, Mapping
 
 import anyio
@@ -61,6 +61,7 @@ def _configure(
         default_headers: Mapping[str, str] | None,
         default_query: Mapping[str, Any] | None,
         credential: Credential | AsyncCredential | None,
+        _strict_response_validation: bool,
     ) -> None:
         if not isinstance(max_retries, int) or isinstance(max_retries, bool) or max_retries < 0:
             raise ValueError("max_retries must be a non-negative integer")
@@ -77,6 +78,7 @@ def _configure(
         self.max_retries = max_retries
         self.default_headers = dict(default_headers or {})
         self.default_query = dict(default_query or {})
+        self._strict_response_validation = _strict_response_validation
 
     def _copy(self, *, raw: bool = False, streaming: bool = False, **overrides: Any) -> Self:
         options = dict(
@@ -88,6 +90,7 @@ def _copy(self, *, raw: bool = False, streaming: bool = False, **overrides: Any)
             default_query=self.default_query,
             credential=self.credential,
             http_client=self._client,
+            _strict_response_validation=self._strict_response_validation,
         )
         options.update(overrides)
         client = type(self)(**options)
@@ -191,19 +194,20 @@ def _retryable(self, request: httpx.Request, response: httpx.Response | None) ->
 
     def _retry_delay(self, retry: int, response: httpx.Response | None) -> float:
         if response is not None:
-            for header, divisor in (("retry-after-ms", 1000), ("retry-after", 1)):
-                value = response.headers.get(header)
-                if value is None:
-                    continue
+            delay = None
+            try:
+                delay = float(response.headers["retry-after-ms"]) / 1000
+            except (KeyError, ValueError):
+                value = response.headers.get("retry-after", "")
                 try:
-                    delay = float(value) / divisor
+                    delay = float(value)
                 except ValueError:
-                    try:
-                        delay = parsedate_to_datetime(value).timestamp() - time.time()
-                    except (ValueError, TypeError, OverflowError):
-                        continue
-                if 0 <= delay <= 60:
-                    return delay
+                    retry_date = parsedate_tz(value)
+                    if retry_date is not None:
+                        delay = mktime_tz(retry_date) - time.time()
+            if delay is not None and delay > 0:
+                # Anthropic caps server-requested waits at the portable sleep limit.
+                return min(delay, 4_294_967.0)
         return min(0.5 * (2 ** min(retry, 10)), 8.0) * (1 - 0.25 * random.random())
 
     def _download_request(self, response: httpx.Response, data: Any) -> httpx.Request:
@@ -237,9 +241,9 @@ def _parse_api_response(
             return None
         data = self._data(response)
         if not page_style:
-            return parse_response(cast_to, data, response)
+            return parse_response(cast_to, data, response, strict=self._strict_response_validation)
         page_cls = AsyncPage if self._is_async else SyncPage
-        page = parse_response(page_cls[cast_to], data, response)
+        page = parse_response(page_cls[cast_to], data, response, strict=self._strict_response_validation)
         page._style = page_style
         page._query = {**self.default_query, **options.get("query", {})}
         client = self._copy() if self._raw_response else self
@@ -261,6 +265,7 @@ def __init__(
         default_query: Mapping[str, Any] | None = None,
         http_client: httpx.Client | None = None,
         credential: Credential | None = None,
+        _strict_response_validation: bool = False,
     ) -> None:
         self._configure(
             pat=pat,
@@ -270,6 +275,7 @@ def __init__(
             default_headers=default_headers,
             default_query=default_query,
             credential=credential,
+            _strict_response_validation=_strict_response_validation,
         )
         if http_client is not None and not isinstance(http_client, httpx.Client):
             raise TypeError("http_client must be an httpx.Client")
@@ -331,7 +337,7 @@ def request(
         request.read()
         response = self._send(request)
         if stream:
-            return Stream(response, cast_to)
+            return Stream(response, cast_to, strict=self._strict_response_validation)
         if binary and not download_link:
             return BinaryAPIResponse(response)
 
@@ -374,6 +380,7 @@ def __init__(
         default_query: Mapping[str, Any] | None = None,
         http_client: httpx.AsyncClient | None = None,
         credential: Credential | AsyncCredential | None = None,
+        _strict_response_validation: bool = False,
     ) -> None:
         self._configure(
             pat=pat,
@@ -383,6 +390,7 @@ def __init__(
             default_headers=default_headers,
             default_query=default_query,
             credential=credential,
+            _strict_response_validation=_strict_response_validation,
         )
         if http_client is not None and not isinstance(http_client, httpx.AsyncClient):
             raise TypeError("http_client must be an httpx.AsyncClient")
@@ -449,7 +457,7 @@ async def request(
         await request.aread()
         response = await self._send(request)
         if stream:
-            return AsyncStream(response, cast_to)
+            return AsyncStream(response, cast_to, strict=self._strict_response_validation)
         if binary and not download_link:
             return AsyncBinaryAPIResponse(response)
 

src/qca/common/_exceptions.py

@@ -66,6 +66,10 @@ class ConflictError(APIStatusError):
     pass
 
 
+class RequestTooLargeError(APIStatusError):
+    pass
+
+
 class UnprocessableEntityError(APIStatusError):
     pass
 
@@ -78,6 +82,10 @@ class InternalServerError(APIStatusError):
     pass
 
 
+class OverloadedError(APIStatusError):
+    pass
+
+
 def status_error(response: httpx.Response) -> APIStatusError:
     try:
         body = response.json()
@@ -91,7 +99,9 @@ def status_error(response: httpx.Response) -> APIStatusError:
         403: PermissionDeniedError,
         404: NotFoundError,
         409: ConflictError,
+        413: RequestTooLargeError,
         422: UnprocessableEntityError,
         429: RateLimitError,
+        529: OverloadedError,
     }.get(response.status_code, InternalServerError if response.status_code >= 500 else APIStatusError)
     return cls(f"Error code: {response.status_code} - {message}", response=response, body=body)