[PULP-1777] Fix postgres 65k param limit on the Copy API

[pedro-psb]

This uses the same approach used in pulp_rpm with query param growth-rate based tests. It's an alternative approach to #1499

Basically, we record every query within the unit test using django utilities, do some data treatment, hook in python's inspect module and we get traceable code (in points where querysets are evaluated) to generated SQL, plus some sql metadata. And among that metadata is query params count.

The test uilities themselves can be improved, but it already catches the bulk of scenarios which can potentially trigger this param limit error. Plus an experimental n+1 detector 👀 (which you didn't have in copy! apparently at least, congratz)

---

• Add growth rate based tests
• Fix linear SQL params count growth rate in copy API
• Add n+1 detector test for copy API and fixes for some of them

Assisted-By: Claude Sonnet 5

📜 Checklist

• Commits are cleanly separated with meaningful messages (simple features and bug fixes should be squashed to one commit)
• A changelog entry or entries has been added for any significant changes
• Follows the Pulp policy on AI Usage
• (For new features) - User documentation and test coverage has been added

See: Pull Request Walkthrough

[pedro-psb]

Look at this comment to see some samples of how that new test machinery works: pulp/pulp_rpm#4538 (comment)

I plan to extract those test utilities somehwere, maybe pulpcore, but IHO it's not a big deal to duplicate it here.
Also, I'd like encourage you to play with it, it might help improving queries on some hot paths.

[pedro-psb]

There was a test hanging, I'll investigate it tomorrow.

[pedro-psb]

SQL Diff (filtered by call made from within find_structured_publish_content for a specific growth profile, forget which one):
https://gist.github.com/pedro-psb/b7781af0b3309c02ea784c4f475a0b32

And a nice read about unnest/arrays (although it talks about inserts, it explains well how the idea behind it in general):
https://www.tigerdata.com/blog/boosting-postgres-insert-performance

[pedro-psb]

I've skipped dealing with this to avoid doing too many real plugin changes in one PR.

If you remove this ignore item and run the unit test you'll see exactly what ORM call is causing a N+1 and you can view the SQL report with the snippet (the unit test error output should be informative enough).

oci-env exec cat /tmp/pytest-artifacts/test_query_count_is_size_invariant.packages_across_distributions.small.sql

I had a quick look and the cause for this N+1 is that the code queries Content filtering by releases, and inside a loop you call release.distribution, which triggers a new Release query call per item to fetch the corresponding distribution.