Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions compiler/rustc_codegen_ssa/src/mir/block.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,14 @@ use rustc_abi::{
};
use rustc_ast as ast;
use rustc_ast::{InlineAsmOptions, InlineAsmTemplatePiece};
use rustc_data_structures::packed::Pu128;
use rustc_hir::attrs::AttributeKind;
use rustc_hir::attrs::lang_items::LangItem;
use rustc_lint_defs::builtin::TAIL_CALL_TRACK_CALLER;
use rustc_middle::mir::interpret::{CTFE_ALLOC_SALT, Scalar};
use rustc_middle::mir::{self, AssertKind, InlineAsmMacro, SwitchTargets, UnwindTerminateReason};
use rustc_middle::ty::layout::{HasTyCtxt, LayoutOf, TyAndLayout, ValidityRequirement};
use rustc_middle::ty::print::{with_no_trimmed_paths, with_no_visible_paths};
use rustc_middle::ty::{self, Instance, Ty, TypeVisitableExt};
use rustc_middle::ty::{self, Instance, ScalarInt, Ty, TypeVisitableExt};
use rustc_middle::{bug, span_bug};
use rustc_session::config::OptLevel;
use rustc_span::{Span, Spanned};
Expand Down Expand Up @@ -456,8 +455,8 @@ impl<'a, 'tcx, Bx: BuilderMethods<'a, 'tcx>> FunctionCx<'a, 'tcx, Bx> {
}
} else if target_iter.len() == 2
&& self.mir[targets.otherwise()].is_empty_unreachable()
&& targets.all_values().contains(&Pu128(0))
&& targets.all_values().contains(&Pu128(1))
&& targets.all_values().contains(&ScalarInt::from(0_u128))
&& targets.all_values().contains(&ScalarInt::from(1_u128))

@saethlin saethlin Aug 14, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the new code here checking if targets contains a u128 here? I would think this code would cause this check to not apply when targets is of type u8, a 1-byte ScalarInt. But surely that's wrong... right?

View changes since the review

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine since the add_target() takes a u128, and the values in SwitchTargets is pub(super), so a u8 probably wouldn't end up here?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh. So even though this change does technically use ScalarInt in the targets, all those ScalarInt are secretly u128, which means that the one-byte overhead is used for a constant 16. I think this change is actually incomplete then, and the API of SwitchTargets would need to be in terms of ScalarInt, not u128. That is a more invasive change, but based on the way @RalfJung wrote the linked issue, should result in some cleanup for users of the API.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. I just took a rough look at how users are using these APIs and found several cases using as u128 or even ScalarInt::to_bits(). I'd take some time to do the cleanup then

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah the entire point of that issue was that the ScalarInt stored here should actually have a size that matches the operand we are working on. :) Then e.g. the interpreter can work with that ScalarInt directly rather than having to go via u128.

{
// This is the really common case for `bool`, `Option`, etc.
// By using `trunc nuw` we communicate that other values are
Expand Down
4 changes: 2 additions & 2 deletions compiler/rustc_middle/src/mir/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1732,11 +1732,11 @@ mod size_asserts {

use super::*;
// tidy-alphabetical-start
static_assert_size!(BasicBlockData<'_>, 144);
static_assert_size!(BasicBlockData<'_>, 152);
static_assert_size!(LocalDecl<'_>, 40);
static_assert_size!(SourceScopeData<'_>, 64);
static_assert_size!(Statement<'_>, 40);
static_assert_size!(Terminator<'_>, 104);
static_assert_size!(Terminator<'_>, 112);

@panstromek panstromek Aug 13, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Terminator size changes had caused regressions before, so this is quite unfortunate, but it's also somewhat mysterious, because the effect is not always there.

I tried to reduce SwitchInt size by using slightly different representation in #159928, but it didn't have as much effect (but I also had to change other Terminator variants to make it smaller, and increase its size in metadata, so those changes could erase the wins from SwitchInt).

There's definitely a way to make SwitchInt represenation more optimal, because right now it store one len redundantly, and lot of accesses require redundant branching, but doing so requires some custom data structure and that might be an overkill.

View changes since the review

static_assert_size!(VarDebugInfo<'_>, 88);
// tidy-alphabetical-end
}
7 changes: 3 additions & 4 deletions compiler/rustc_middle/src/mir/syntax.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@

use rustc_abi::{FieldIdx, VariantIdx};
use rustc_ast::{InlineAsmOptions, InlineAsmTemplatePiece, Mutability};
use rustc_data_structures::packed::Pu128;
use rustc_hir::CoroutineKind;
use rustc_hir::def_id::DefId;
use rustc_index::IndexVec;
Expand All @@ -18,7 +17,7 @@ use smallvec::SmallVec;
use super::{BasicBlock, Const, Local, UserTypeProjection};
use crate::mir::coverage::CoverageKind;
use crate::ty::adjustment::PointerCoercion;
use crate::ty::{self, GenericArgsRef, List, Region, Ty, UserTypeAnnotationIndex};
use crate::ty::{self, GenericArgsRef, List, Region, ScalarInt, Ty, UserTypeAnnotationIndex};

/// Represents the "flavors" of MIR.
///
Expand Down Expand Up @@ -977,7 +976,7 @@ pub enum BackwardIncompatibleDropReason {
pub struct SwitchTargets {
/// Possible values. For each value, the location to branch to is found in
/// the corresponding element in the `targets` vector.
pub(super) values: SmallVec<[Pu128; 1]>,
pub(super) values: SmallVec<[ScalarInt; 1]>,

/// Possible branch targets. The last element of this vector is used for
/// the "otherwise" branch, so `targets.len() == values.len() + 1` always
Expand Down Expand Up @@ -1746,6 +1745,6 @@ mod size_asserts {
static_assert_size!(PlaceElem<'_>, 24);
static_assert_size!(Rvalue<'_>, 40);
static_assert_size!(StatementKind<'_>, 16);
static_assert_size!(TerminatorKind<'_>, 80);
static_assert_size!(TerminatorKind<'_>, 88);
// tidy-alphabetical-end
}
18 changes: 9 additions & 9 deletions compiler/rustc_middle/src/mir/terminator.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@
use std::slice;

use rustc_ast::InlineAsmOptions;
use rustc_data_structures::packed::Pu128;
use rustc_hir::attrs::AttributeKind;
use rustc_hir::attrs::lang_items::LangItem;
use rustc_macros::{StableHash, TyDecodable, TyEncodable, TypeFoldable, TypeVisitable};
use smallvec::{SmallVec, smallvec};
use thin_vec::ThinVec;

use super::*;
use crate::ty::ScalarInt;

impl SwitchTargets {
/// Creates switch targets from an iterator of values and target blocks.
Expand All @@ -19,15 +19,15 @@ impl SwitchTargets {
/// `goto otherwise;`.
pub fn new(targets: impl Iterator<Item = (u128, BasicBlock)>, otherwise: BasicBlock) -> Self {
let (values, mut targets): (SmallVec<_>, SmallVec<_>) =
targets.map(|(v, t)| (Pu128(v), t)).unzip();
targets.map(|(v, t)| (ScalarInt::from(v), t)).unzip();
targets.push(otherwise);
Self { values, targets }
}

/// Builds a switch targets definition that jumps to `then` if the tested value equals `value`,
/// and to `else_` if not.
pub fn static_if(value: u128, then: BasicBlock, else_: BasicBlock) -> Self {
Self { values: smallvec![Pu128(value)], targets: smallvec![then, else_] }
Self { values: smallvec![ScalarInt::from(value)], targets: smallvec![then, else_] }
}

/// Inverse of `SwitchTargets::static_if`.
Expand All @@ -36,7 +36,7 @@ impl SwitchTargets {
if let &[value] = &self.values[..]
&& let &[then, else_] = &self.targets[..]
{
Some((value.get(), then, else_))
Some((value.to_u128(), then, else_))
} else {
None
}
Expand Down Expand Up @@ -72,12 +72,12 @@ impl SwitchTargets {

/// Returns a slice with all considered values (not including the fallback).
#[inline]
pub fn all_values(&self) -> &[Pu128] {
pub fn all_values(&self) -> &[ScalarInt] {
&self.values
}

#[inline]
pub fn all_values_mut(&mut self) -> &mut [Pu128] {
pub fn all_values_mut(&mut self) -> &mut [ScalarInt] {
&mut self.values
}

Expand All @@ -92,7 +92,7 @@ impl SwitchTargets {
/// Adds a new target to the switch. Panics if you add an already present value.
#[inline]
pub fn add_target(&mut self, value: u128, bb: BasicBlock) {
let value = Pu128(value);
let value = ScalarInt::from(value);
if self.values.contains(&value) {
bug!("target value {:?} already present", value);
}
Expand All @@ -108,15 +108,15 @@ impl SwitchTargets {
}

pub struct SwitchTargetsIter<'a> {
inner: iter::Zip<slice::Iter<'a, Pu128>, slice::Iter<'a, BasicBlock>>,
inner: iter::Zip<slice::Iter<'a, ScalarInt>, slice::Iter<'a, BasicBlock>>,
}

impl<'a> Iterator for SwitchTargetsIter<'a> {
type Item = (u128, BasicBlock);

#[inline]
fn next(&mut self) -> Option<Self::Item> {
self.inner.next().map(|(val, bb)| (val.get(), *bb))
self.inner.next().map(|(val, bb)| (val.to_u128(), *bb))
}

#[inline]
Expand Down
2 changes: 1 addition & 1 deletion compiler/rustc_mir_dataflow/src/impls/initialized.rs
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ impl<'tcx> MaybePlacesSwitchIntData<'tcx> {
// On each call to this closure `find` only consumes part of
// the `discriminants` iterator.
discriminants
.find(|(_, discr)| discr.val == value.get())
.find(|(_, discr)| discr.val == value.to_u128())
.expect("SwitchInt vals should match a variant")
.0
})
Expand Down
2 changes: 1 addition & 1 deletion compiler/rustc_mir_transform/src/ssa_range_prop.rs
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,7 @@ impl<'tcx> MutVisitor<'tcx> for RangeSet<'tcx, '_, '_> {
&& self.unique_predecessors.contains(otherwise.block)
{
assert_ne!(location.block, otherwise.block);
let range = if val.get() == 0 {
let range = if val.to_u128() == 0 {
WrappingRange { start: 1, end: 1 }
} else {
WrappingRange { start: 0, end: 0 }
Expand Down
Loading