Conversation
aws/sagemaker-python-sdk#4871 has been resolved
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1350 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 171 171
Lines 11648 11648
Branches 1538 1538
=========================================
Hits 11648 11648 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
This change becomes tricky because cloudpickle explicitly doesn't support serializing with one version and deserializing with a different version. If we unpin it, we create the possibility of mismatched versions between a user's client (wherever they are running their code with the @hybrid_job decorator), and the hybrid job container where it runs.
For a specific example: after the job container upgrades to an SDK version with this change, it will get the latest cloudpickle version (3.x), but users of older SDK versions or with older environments will be using old cloudpickle (2.x). This is not guaranteed to work (and in practice we've seen that it often doesn't).
Is there a reason that we truly want/need a newer cloudpickle version? If so, we need to somehow safeguard against the above scenario.
aws/sagemaker-python-sdk#4871 has been resolved
Issue #, if available:
Description of changes:
Testing done:
Merge Checklist
Put an
xin the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.General
Tests
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.