Repository navigation
gh-160601: Handle surrogate characters in sampling profiler binary output - #160615
Open
zolaski333 wants to merge 1 commit into
Open
zolaski333 wants to merge 1 commit into
zolaski333 wants to merge 1 commit into
Conversation
This was referenced Oct 28, 2026
Merged
ChrisBuilds
added a commit
to ChrisBuilds/terminaltexteffects
that referenced
this pull request
Oct 28, 2026
## Resulting behavior The Profile workflow no longer fails when a sampled frame name can't be encoded. `tools/perf/sample_profile.py` now checks each sample's file and function names before calling the binary writer, and drops a sample with lone surrogates without writing it. It still counts and reports dropped samples as before, so the "Profiler dropped samples" warning is unchanged. The #354 wrapper called the writer first and caught the `UnicodeEncodeError`. That can't work, because CPython 3.15's `BinaryWriter.write_sample` (`Modules/_remote_debugging/module.c`) sets `state = BINARY_WRITER_BROKEN` after any failed write and refuses every later sample with `ValueError: Writer is broken`. That's the failure on `main` at 7bef9a9 and de137a5: one dropped sample, then the traceback. The name check returns early for ASCII names (`str.isascii()`), so the regex only runs on the rare non-ASCII names. Closes #361 ## Validation - `tests/test_perf_sample_profile.py`: the fake writer now behaves like CPython's and refuses every sample after a failed one. On Python 3.15, the test fails against the old wrapper with `ValueError: Writer is broken` and passes with this change. The name-detection test also passes on 3.14. - Ruff format and lint, and Pyright, on the touched files: passed. - Manual Profile run on this branch with the defaults, covering `all-large-iterator`, where #354's merge commit failed: https://github.com/ChrisBuilds/terminaltexteffects/actions/runs/40764532996. It passed, but no sampled name needed dropping this time, so it confirms the wrapper runs cleanly without exercising the drop. - I couldn't run it end to end locally: macOS needs root to attach the sampling profiler. ## Limitations The bad names appear only in some runs, so one green Profile run doesn't prove the fix. The unit test reproduces the writer's broken state directly. The wrapper can be removed once the Python used for profiling includes python/cpython#160615. ## Review checklist - [x] Assignee, labels, and release milestone match the linked issue. - [x] The linked issue's acceptance criteria are met. - [x] Changes stay within the agreed scope. - [x] Relevant documentation and generated completions are current. - [x] Changelog fragment added for user-facing changes, or the `no-changelog` label applied. - [x] Required CI checks pass for the latest revision.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Encode sampled filenames and function names using UTF-8 with
backslashreplace, preventingUnicodeEncodeErrorwhen they contain surrogate characters. The string table remains valid UTF-8 and compatible with the existing reader.Add regression coverage for surrogate characters in both filenames and function names, including mixed Unicode names and repeated samples.
Validation on Windows:
test_profiling: passed.-R 3:3: passed, no leaks detected.patchcheckandgit diff --check: passed.Zstd compression was unavailable in this Windows build.
profiling.samplingin--binarymode raises aUnicodeEncodeErrorwhile collecting the samples #160601