Fix distributedType -> distributed_type typo in downcast_bf16 guard - #4149
Open
uttam12331 wants to merge 1 commit into
Open
Fix distributedType -> distributed_type typo in downcast_bf16 guard#4149uttam12331 wants to merge 1 commit into
uttam12331 wants to merge 1 commit into
Conversation
The downcast_bf16 validation in Accelerator.__init__ references `self.state.distributedType`, which does not exist -- the attribute is `distributed_type` (used on the line above at 520 and throughout state.py). When mixed_precision != "bf16" and downcast_bfloat is set, the guard is reached and raises `AttributeError: ... has no attribute 'distributedType'` instead of the intended, clear ValueError.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
downcast_bf16validation inAccelerator.__init__referencesself.state.distributedType, which does not exist — the attribute isdistributed_type:self.stateis anAcceleratorState, whose attribute isdistributed_type(used one line above at ~520:self.state.distributed_type == DistributedType.FSDP, and throughoutstate.py).distributedTypeappears only on this line in the entiresrc/tree.Impact
Because of short-circuit evaluation, this line is reached when
mixed_precision != "bf16"anddowncast_bfloatis set. In that case — a user enablingdowncast_bf16withoutmixed_precision="bf16"— the code raises:instead of the intended, actionable
ValueErrorthat explains the misconfiguration.Fix
One-token typo fix; the guard now raises the intended
ValueError.(Happy to add a regression test for the
downcast_bf16-without-bf16path if you'd like — let me know the preferred fixture for constructing that state.)