Hardening security and stability - #65
Merged
Merged
Conversation
…hing - Neutralize multi-pass URL encoding bypasses (CWE-116) and catch timeout float overflows. - Add _deep_sanitize to IdempotencyGuard against cyclic references and recursion overflows. - Preserve and restore file stream seek pointers during payload fingerprint hashing. - Block executable tags and inline event handlers in SpamGuard. - Update unit tests covering proxy initialization, overflow validation, and tag blocks.
…mat injection - Fallback to list representation when redacted set elements become unhashable. - Wrap custom object attribute inspection in defensive exception boundaries. - Preserve record.args tuple, dict, and list structures to prevent string formatting crashes. - Increase MAX_REDACTION_DEPTH to 5 and add support for frozenset. - Add regression tests covering libFuzzer format operator payloads.
…s safely - Implement seek() and tell() on ChunkedStreamer with explicit EOF state tracking. - Validate that chunk_size is a strictly positive integer. - Return a shallow copy of files from MailgunMessageBuilder.build(). - Add default=str to json.dumps across message and template builders. - Add unit tests verifying ChunkedStreamer EOF, seek/tell, and invalid chunk sizes.
…attributes - Canonicalize domain aliases via DOMAIN_ALIASES and preserve literal '@' in credentials logins. - Explicitly block dynamic attribute lookups for 'config' and 'auth' in Client.__getattr__. - Add unit test asserting Client.__getattr__ raises AttributeError on internal config and auth lookups.
…eam pagination - Retain original filter types (tuple, set, list, int, float, bool) across cursor iterations in Endpoint and AsyncEndpoint. - Stop pagination cleanly when response items are empty or response payload is not a dictionary. - Add unit tests for synchronous and asynchronous stream pagination type casting and non-dict response guards.
- Guard route deletion against empty items list to avoid IndexError on clean test domains. - Safely parse sender address in sync route tests via email.utils.parseaddr. - Add region_data assertion to sync and async user payload tests.
…le invariants - Add property tests for IdempotencyGuard stream seek pointer restoration. - Add property tests for RedactingFilter args type preservation during string formatting. - Add webhook replay TTL window property tests. - Implement MailgunStateSequenceMachine for end-to-end stateful lifecycle testing.
…ful fuzzer - Append LibFuzzer dictionary tokens targeting XML/CDATA, formatting specifiers, and encodings. - Add stream pointer verification, cyclic variable payloads, null paging cursors, and timeout chaos actions to fuzz_stateful_client.
…object attributes - Wrap dict.items() and record.__dict__.keys() in list() calls during deep redaction to prevent dictionary modification errors during iteration. - Guard model_dump and __dict__ attribute access in defensive try-except blocks to catch dynamic property lookup failures. - Return safe fallback directly when str(data) raises during stringification. - Add regression tests for crashing fuzz payloads and exploding __repr__/__str__ objects.
- Append malformed HTML doctype tokens and crash reproducer byte sequences to fuzz.dict.
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
… environment files, fix linter issues
… domain sanitization - Convert raw HTTPStatusError into ApiError during async lazy pagination. - Enforce ISO-8859-1 (Latin-1) wire encoding and CRLF boundary checks in SecurityGuard.sanitize_headers(). - Reject domains with null bytes or unhandled control characters in SecurityGuard.normalize_domain(). - Harden handle_address_validate() against non-sequence URL key payloads.
- Add field validator on subject to reject CRLF injection sequences (CWE-113). - Clean up redundant comments and streamline model configuration.
…y seeds - Add 5-second execution timeout cap to manage.sh fuzz_all(). - Update fuzz.dict with newly harvested edge cases and control byte tokens. - Add concurrency and connection pool stress testing to fuzz_async_client.py. - Prevent exponential backoff blocking and handle Retry-After edge cases in fuzz_async_evil_server.py. - Modernize test harnesses across builders, handlers, endpoints, webhooks, and stateful clients.
…sources - Set reportMissingModuleSource to none in pyproject.toml. - Silence stub warnings for untyped third-party packages in pre-commit environments.
- Document frozenset support in RedactingFilter. - Note requests >=2.33.0 dependency synchronization.
…ork:mailgun/mailgun-python into fix/v1.9.1-hardening-and-stability
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Links:
Jira
Actions:
Security & Guardrails:
ALLOWED_SCHEMESvalidation (https,http) inSecurityGuard.validate_mailgun_url()and extended trusted host rules to cover.mailgun.com.SecurityGuard.sanitize_domain()to recursively evaluateunquote()across up to 3 passes, apply NFKC Unicode normalization, and strip CRLF/slash patterns prior to evaluating directory traversal sequences (..).SecurityGuard.verify_webhook()to 900 seconds (15 minutes), rejecting expired requests while allowing<= 0to bypass timestamp evaluation during static unit testing.SpamGuardwith_BLOCKED_TAGS(iframe,object,embed,applet) and automated regex detection for inline DOM event handlers (on*), adding pre-parsing payload length validation prior to feeding the HTML parser.MAX_REDACTION_DEPTHto 5 inRedactingFilterand addedfrozensetsupport. Preservedrecord.argstypes (tuple, dict, list) to prevent string formatting crashes, wrapped object inspection in defensive guards, and provided fallback list representations when redacted set elements become unhashable._deep_sanitize()with a 50-level depth threshold toIdempotencyGuardto eliminate circular reference crashes and recursion overflows during fingerprint calculations.SecurityGuard.sanitize_headers(), guardingsys.audittelemetry invocations withsys in sys.modules.Bug Fixes & Network Resilience:
IdempotencyGuard.generate_key()to recordstream.tell()before reading file payloads and restore the stream offset viaseek()after hashing, preventing stream exhaustion during retries.seek()andtell()onChunkedStreamer, enforced strictly positivechunk_sizebounds, and introduced explicit_eofstate tracking to prevent duplicate reads._cast_query_param()to preserve user filter collection types (int,float,bool,list,tuple,set) during cursor pagination, and added guards for missingnextlinks or non-dictionary API responses.default=strtojson.dumps()acrossMailgunMessageBuilder,MailgunTemplateBuilder, andBaseEndpointto prevent serialization crashes on custom objects (UUID, datetime).list(self._files)) inMailgunMessageBuilder.build()to prevent external consumers from mutating internal builder state.configandauthinClient.__getattr__(), raisingAttributeErrorinstead of constructing bad endpoint paths.DOMAIN_ALIASES) directly inhandle_domains()and preserved literal@delimiters during credential path generation.OverflowErrorhandling during float conversion inSecurityGuard.sanitize_timeout(), mapping out-of-capacity numbers to standardValueErrorexceptions.SecurityGuard.normalize_domain()to partition email addresses and apply IDNA Punycode encoding specifically to the domain segment.Architecture & Developer Experience (DX):
routes_matchunder v3 endpoint routing, mappedreputationanalytics_v2under v2 prefix routes, and registered deprecation warnings for the legacyv1/spamtrapsAPI family[.requests >=2.33.0, decoupled channel-pinnedconda-forge::httpx2, and synchronized.pre-commit-config.yamldependency constraints.missing-trailing-commaselector frompyproject.tomlto restoremdformat-ruffcompatibility.Testing, CI/CD & Benchmarks:
fuzz_stateful_client.pywith attachment seek pointer validation, circular custom variable injection, null cursor pagination shocks, and timeout boundary tests.tests/property/tests.pywith invariant checks for idempotency seek pointers, redacting filter args preservation, path segment sanitization idempotency, and webhook replay windows.__repr__/__str__custom objects intest_regression.py.PERFORMANCE.md.Verification & Testing:
To verify these changes locally, ensure your environment variables (
APIKEYandDOMAIN, and others) are set, then run the following commands:1. Run the Unit Test Suite (Fast):
Validates the core routing logic, new guardrails (
SpamGuard,IdempotencyGuard),RetryPolicy, and strictPydanticpayload schemas.2. Run the Live Routing Meta-Test (No state mutation):
Proves the SDK correctly constructs URLs for all supported endpoints by hitting live Mailgun servers (expects 200, 400, 401, or 403 responses; tests fail if the Python SDK crashes or generates a 404 bad route).
3. Run the Full Integration Suite (State mutation):
Executes end-to-end flows against your Sandbox domain (creates/deletes real resources).
4. Execute the Interactive Smoke Test:
Runs the executable documentation script demonstrating cross-version routing and payload serialization.
5. Run the Fuzzing Suite (Security Saturation):
Executes Atheris mutation coverage across core handlers, parsers, and client lifecycles.
6. Run the Performance & Cold-Boot Benchmarks:
Validates the new O(1) routing dispatch and
__slots__memory optimizations using our unified DX script.