zephyr-cp tests: read serial output in chunks instead of one byte at a time - #11353
Open
lynt-smitka wants to merge 2 commits into
Open
zephyr-cp tests: read serial output in chunks instead of one byte at a time#11353lynt-smitka wants to merge 2 commits into
lynt-smitka wants to merge 2 commits into
Conversation
SerialSaver's reader thread read one byte per iteration and appended it to self.all_output. That store goes through STORE_ATTR, so CPython's in-place unicode concatenation does not apply and every byte copies the whole string. The bsim tests run the BabbleSim phy at -v=9 through the same reader, which is about 1.1 MB per test. Measured here, 993 KB of that costs 9.4 s in the reader against a test budget of 10 to 30 s of wall clock, and the phy stalls whenever the reader falls more than one pipe buffer behind. Read what is already buffered instead. The two bsim test files go from 144 s to 26 s on an idle host, and from 340 s with failures to 57 s green under CPU oversubscription.
It was the only bsim test below the default 10 s, and it is the first one to miss its deadline when the host is busy: with 32 spinning processes on a 4 core machine it failed both runs at 5 s and passed all three at 10 s. The timeout is only an upper bound, wait_until_done() returns as soon as the device exits, so the larger budget costs nothing when the test passes. The full bsim suite runs in 106 s either way.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SerialSaver's reader thread read one byte at a time and appended it to an attribute, so each byte copied the whole string: 993 KB took 9.4 s, 1.99 MB took 35.6 s. Each bsim test pipes about 1.1 MB of phy output through it, against a wall-clock budget.
It now reads whatever is already buffered. The bsim suite (118 tests) goes from 554.7 s to 109.6 s idle. The two files that fail most on CI go from 144 s to 26 s, and under load from 340 s with 2 of 3 runs failing to 57 s, all green. This may also be behind the intermittent
Low level communication with phy failed errors.The second commit drops
duration(5)from one bsim test, the only one below the default 10 s. It failed at 5 s under load and passed at 10 s.