From 0c282cab2724969f0840a4c9046daad8ace876f4 Mon Sep 17 00:00:00 2001 From: yakkala-pooja Date: Mon, 7 Sep 2026 23:06:16 -0400 Subject: [PATCH] Make --codegen=linker and --sysroot path-mapping-aware for a cc_toolchain-provided linker cc_common.get_tool_for_action() returns the C++ toolchain's linker as a plain string, and cc_common.get_memory_inefficient_command_line() returns its link args (including a --sysroot= value) as plain strings too. Passing either straight into Args means Bazel's path mapping (--experimental_output_paths=strip) has nothing it can rewrite, since only a File threaded through Args is eligible -- so a hermetic C++ toolchain whose linker or sysroot is a generated Bazel artifact ends up with a stale, un-rewritten configuration segment in the rustc command line, and linking fails when the Rustc action runs against the stripped bazel-out/cfg layout. get_linker_and_args now recovers the backing File for the linker (searching cc_toolchain.all_files for the string cc_common.get_tool_for_action() returned, including the case where the tool lives inside a directory/tree artifact) and threads it through rustc_flags.add/add_all instead of the raw string, mirroring the map_each/format_each pattern this file already uses for the Rust toolchain's own sysroot. The Rust-toolchain-provided linker (toolchain.linker) already had a File on hand for free, so that branch is fixed the same way. link_args gets the same treatment for a --sysroot= entry, via a small wrapper that leaves every other entry, and any entry that fails to resolve, untouched. The resolution is skipped up front for any path that doesn't start with bazel-out/: a system-absolute path (the common, non-hermetic case), an external-repository path, and a plain source-tree path are all already configuration-independent, so path mapping has nothing to rewrite for them and the scan would be pure overhead. An exact match is tried first; a fallback matches on the part of the path after bazel-out//bin/ instead, for a version where cc_toolchain's own files are reported under a different configuration segment than cc_common.get_tool_for_action()'s string for the very same toolchain -- confirmed via a temporary debug dump against the repo's minimum-supported Bazel version (7.4.1) in CI, which showed the exact same toolchain's tool resolving to a target-config path from get_tool_for_action() but an exec-config path in cc_toolchain.all_files. Using the matched File's own path to render the flag either way, not the original string, means whichever configuration Bazel actually materializes that File under is what ends up on the command line, so the two disagreeing on the configuration segment does not matter. construct_arguments now also tracks the File(s) it resolved this way on the returned args struct as extra_action_inputs, and rustc_compile_action merges that into the Rustc/RustcMetadata actions' inputs. Passing a File through Args puts it on the command line but does not by itself register it as an action input, and collect_inputs' own cc_toolchain.linker_files() mechanism did not reliably do this in every configuration this PR was tested against. The whole linker_files() depset is pulled in explicitly too -- using the same hasattr(cc_toolchain, "_linker_files") version-compat check collect_inputs already uses for that same method -- but only when the linker or a --sysroot= entry actually needed the config-agnostic or directory-relative fallback match in _resolve_tool_file (its resolved File's own path landing on something other than the original string is exactly that signal), not on every hermetic-toolchain build: an exact match has proven sufficient on its own on every Bazel version this needed it, so the extra flatten is reserved for the specific case that isn't true. Added test/unit/path_mapped_linker: a minimal cc_toolchain whose linker and sysroot are genrule outputs (so their real location is a config-dependent bazel-out/... path, unlike a normal system toolchain), registered via --extra_toolchains with linker_preference forced to "cc". Several things only surfaced against real CI, not local testing, and are addressed: - analysis_test_transition refuses to set --experimental_* options, so the test cannot force --experimental_output_paths=strip on itself. The assertions check only that --codegen=linker= and the --sysroot= link-arg name the right file, not the mapped bazel-out/cfg/... prefix specifically; the repo's own "Path Mapping Linux/RBE/MacOS" CI jobs already run the whole suite, this test included, with the flag set, which is where that stronger check actually happens. Verified manually both ways: run normally the test passes trivially; run with --experimental_output_paths=strip on the command line, temporarily reverting just the resolution step in rustc.bzl makes it fail, showing the real, un-rewritten configuration segment in place of bazel-out/cfg/ for exactly the flag that was reverted -- confirmed independently for the linker and the sysroot. - The fake toolchain, registered via --extra_toolchains with no compatibility constraints, was a candidate for every C++ toolchain resolution in the build. Under bazel coverage specifically (which forces a fresh, uncached build of exec-configuration tools), it was being selected to "link" util/process_wrapper -- a Rust binary with no relation to this test -- and since the fake linker was a no-op script, that build failed with "output ... was not created" on both Linux/RBE and macOS CI. Fixed by giving fake_cc_toolchain a target_compatible_with constraint satisfied only by a dedicated fake_platform (extending the real host platform via parents), and transitioning --platforms to it alongside --extra_toolchains: the fake toolchain now only ever matches this test's own target configuration. - analysistest never executes the target-under-test's own actions (it only inspects the analysis-time action graph), but bazel coverage does -- so once the fake linker was actually the one selected for this test's own rust_binary (correctly, now that the previous leak is fixed), its being a pure no-op became a real problem: Bazel requires a declared output to actually exist, and the no-op script produced none. The fake linker (test/unit/path_mapped_linker/fake_gcc_template.sh, copied into place by a genrule so its exec path stays under bazel-out/) parses "-o " and "/OUT:" and creates whichever one it is given, via ": > \"\$out\"", a shell builtin, not the external touch: the sandbox PATH a linker is invoked under is minimal and does not reliably have it (observed directly as "touch: not found" in the linker's own stderr, from CI). - Fixing the toolchain leak then surfaced the bug this PR actually fixes, one level up: with the fake toolchain correctly scoped to just this test's own rust_binary, its Rustc action failed with "linker bazel-out/.../fake_gcc not found" under bazel coverage on Bazel 7.4.1 specifically -- the file was on the command line but not materialized in the sandbox. Root-caused (see above) to cc_toolchain resolving to a different configuration than get_tool_for_action()'s own string on that Bazel version; extra_action_inputs plus the config-agnostic suffix match together close the gap. Testing: reproduced the toolchain-leak scenario locally with bazel coverage over this test plus util/process_wrapper together and confirmed process_wrapper falls through to the real toolchain; verified the fake linker script's argument parsing directly in isolation, including with an empty PATH to simulate the restricted sandbox environment that surfaced the touch issue. A full end-to-end Rust build could not be exercised on the development machine at all (no MSVC installed, unrelated to this change), and swapping to Bazel 7.4.1 via bazelisk hit the same MSVC gap rather than reaching the Linux-specific failures, so the minimum-Bazel- version-specific issues above were diagnosed from real CI logs -- one via a temporary print-based debug dump added to a throwaway commit, run against CI, then removed once it gave the answer -- rather than independently reproduced end-to-end. Fixes #4250 Assisted-by: Claude (Anthropic) --- cargo/private/cargo_build_script.bzl | 2 +- extensions/bindgen/private/bindgen.bzl | 2 +- rust/private/rustc.bzl | 239 +++++++++++++++++- test/unit/path_mapped_linker/BUILD.bazel | 90 +++++++ .../path_mapped_linker/fake_gcc_template.sh | 27 ++ test/unit/path_mapped_linker/main.rs | 1 + .../path_mapped_linker_test.bzl | 117 +++++++++ 7 files changed, 468 insertions(+), 10 deletions(-) create mode 100644 test/unit/path_mapped_linker/BUILD.bazel create mode 100644 test/unit/path_mapped_linker/fake_gcc_template.sh create mode 100644 test/unit/path_mapped_linker/main.rs create mode 100644 test/unit/path_mapped_linker/path_mapped_linker_test.bzl diff --git a/cargo/private/cargo_build_script.bzl b/cargo/private/cargo_build_script.bzl index 12a2271cd9..06443e626f 100644 --- a/cargo/private/cargo_build_script.bzl +++ b/cargo/private/cargo_build_script.bzl @@ -498,7 +498,7 @@ def _cargo_build_script_impl(ctx): cc_toolchain, feature_configuration = find_cc_toolchain(ctx) else: cc_toolchain, feature_configuration = None, None - linker, _, link_args, linker_env = get_linker_and_args(ctx, "bin", toolchain, cc_toolchain, feature_configuration, None) + linker, _, _, link_args, linker_env = get_linker_and_args(ctx, "bin", toolchain, cc_toolchain, feature_configuration, None) env.update(**linker_env) env["LD"] = linker env["LDFLAGS"] = " ".join(_pwd_flags(link_args)) diff --git a/extensions/bindgen/private/bindgen.bzl b/extensions/bindgen/private/bindgen.bzl index b5f9ec1cac..c6c29d8639 100644 --- a/extensions/bindgen/private/bindgen.bzl +++ b/extensions/bindgen/private/bindgen.bzl @@ -414,7 +414,7 @@ def _rust_bindgen_impl(ctx): for define in ctx.attr.cc_lib[CcInfo].compilation_context.defines.to_list(): args.add("-D" + define) - _, _, _, linker_env = get_linker_and_args(ctx, "bin", rust_toolchain, cc_toolchain, feature_configuration, None) + _, _, _, _, linker_env = get_linker_and_args(ctx, "bin", rust_toolchain, cc_toolchain, feature_configuration, None) env.update(**linker_env) # Set the dynamic linker search path so that clang uses the libstdcxx from the toolchain. diff --git a/rust/private/rustc.bzl b/rust/private/rustc.bzl index d447c97303..d7069ba440 100644 --- a/rust/private/rustc.bzl +++ b/rust/private/rustc.bzl @@ -375,6 +375,157 @@ def get_cc_user_link_flags(ctx): """ return ctx.fragments.cpp.linkopts +def _bazel_out_relative_suffix(path): + """Returns the config-independent part of a `bazel-out//bin/...` path. + + Args: + path (str): A path, typically a `File.path` or the string + `cc_common.get_tool_for_action()` returned. + + Returns: + str or None: Everything after `bazel-out//bin/`, or `None` + if `path` isn't shaped like that (e.g. it's not under `bazel-out/` + at all, or uses an output root other than `bin`). + """ + parts = path.split("/", 3) + if len(parts) == 4 and parts[0] == "bazel-out" and parts[2] == "bin": + return parts[3] + return None + +def _resolve_tool_file(tool_path, files): + """Finds the `File` backing a tool path returned by `cc_common.get_tool_for_action()`. + + That function returns a plain string, which Bazel's path mapping + (`--experimental_output_paths=strip`) cannot rewrite -- only a path that + reaches `Args` as a `File` object is eligible. This recovers the + underlying `File` so the caller can pass it back through `Args` instead + and regain path mapping, including when the tool lives inside a + directory (tree) artifact rather than being its own `File`. + + Only a path under `bazel-out/` can possibly need this: a system-absolute + path (the common case -- an auto-configured, non-hermetic toolchain), + an external-repository path, and a plain source-tree path are all + already configuration-independent, so path mapping has nothing to + rewrite for them. Bailing out before scanning `files` keeps this a + no-op for every build that isn't using a hermetic, Bazel-generated + toolchain. + + An exact match is tried first; on a version where `cc_toolchain`'s own + files are reported under a different configuration segment than + `tool_path` uses for the very same toolchain (observed against Bazel + 7.4.1 -- the same toolchain, resolved for two different configurations, + e.g. one target-config and one exec-config copy), falls back to + matching on the part of the path after `bazel-out//bin/`. Using + the matched `File`'s own path to render the flag either way (not + `tool_path`) means whichever configuration Bazel actually materializes + that File under is what ends up on the command line, so the two + disagreeing on the configuration segment doesn't matter. + + Args: + tool_path (str): The exec path returned by `cc_common.get_tool_for_action()`. + files (depset[File]): Toolchain files to search, e.g. `cc_toolchain.all_files`. + + Returns: + tuple: (File, str or None) -- the matching `File`, and, when the tool lives + inside a directory artifact, the relative path beneath it (`None` for an + exact match). `(None, None)` when no match is found. + """ + if not tool_path.startswith("bazel-out/"): + return None, None + + tool_suffix = _bazel_out_relative_suffix(tool_path) + + best_dir = None + suffix_match = None + for f in files.to_list(): + if f.path == tool_path: + return f, None + if f.is_directory and tool_path.startswith(f.path + "/"): + if best_dir == None or len(f.path) > len(best_dir.path): + best_dir = f + elif not f.is_directory and suffix_match == None and tool_suffix != None: + if _bazel_out_relative_suffix(f.path) == tool_suffix: + suffix_match = f + if best_dir: + return best_dir, tool_path[len(best_dir.path) + 1:] + if suffix_match: + return suffix_match, None + return None, None + +def _tool_file_path(entry): + """`map_each` callback rendering a tool inside a directory artifact. + + `entry` is a `struct(file, suffix)` produced by `_resolve_tool_file`. + Accessing `entry.file.path` here (inside `map_each`) is what makes the + directory artifact's prefix eligible for Bazel's path mapping; `suffix` + is the relative path beneath it, which does not vary by configuration. + + Args: + entry (struct): A `struct(file, suffix)` pair. + + Returns: + str: `entry`'s full tool path. + """ + return entry.file.path + "/" + entry.suffix + +def _wrap_sysroot_link_args(link_args, cc_toolchain): + """Wraps a resolvable `--sysroot=` entry for path-mapping-aware rendering. + + `link_args` comes from `cc_common.get_memory_inefficient_command_line()` as + plain strings. When a `--sysroot=` value points at a toolchain artifact + under `bazel-out/`, wrap it in a `struct` so `_map_each_link_arg` can + render it through the underlying `File` (see `_resolve_tool_file`); + every other entry, and any entry that fails to resolve, passes through + unchanged. + + Args: + link_args (list): Flattened linker command line flags. + cc_toolchain (CcToolchainInfo or None): The current C++ toolchain, if any. + + Returns: + tuple: (list, bool) -- `link_args` with resolvable `--sysroot=` + entries wrapped, and whether any of them needed a config-agnostic + or directory-relative match rather than an exact one (see + `_resolve_tool_file`), signalling to the caller that the + broader `cc_toolchain.linker_files()` safety net may be needed. + """ + if not cc_toolchain: + return link_args, False + + wrapped = [] + needs_fallback = False + for arg in link_args: + if arg.startswith("--sysroot="): + path = arg[len("--sysroot="):] + matched_file, suffix = _resolve_tool_file(path, cc_toolchain.all_files) + if matched_file: + effective = matched_file.path if suffix == None else matched_file.path + "/" + suffix + if effective != path: + needs_fallback = True + wrapped.append(struct(prefix = "--sysroot=", file = matched_file, suffix = suffix)) + continue + wrapped.append(arg) + return wrapped, needs_fallback + +def _map_each_link_arg(arg): + """`map_each` callback for the (mostly plain-string) link args list. + + A plain string passes through unchanged. A `struct(prefix, file, suffix)` + from `_wrap_sysroot_link_args` is rendered through `file.path` here, + inside `map_each`, so Bazel's path mapping applies to it. + + Args: + arg (str or struct): One entry from a `_wrap_sysroot_link_args()` result. + + Returns: + str: The flag's value (`format_each` in the caller adds the + `--codegen=link-arg=` prefix on top of this). + """ + if type(arg) == "struct": + path = arg.file.path if arg.suffix == None else arg.file.path + "/" + arg.suffix + return arg.prefix + path + return arg + def get_linker_and_args(ctx, crate_type, toolchain, cc_toolchain, feature_configuration, rpaths, add_flags_for_binary = False): """Gathers cc_common linker information @@ -391,6 +542,12 @@ def get_linker_and_args(ctx, crate_type, toolchain, cc_toolchain, feature_config Returns: tuple: A tuple of the following items: - (str): The tool path for given action. + - (File or struct or None): The linker as a `File` (or, for a tool inside + a directory artifact, a `struct(file, suffix)` -- see `_tool_file_path`) + for path-mapping-aware rendering, when resolvable. `None` when the tool + path could not be associated with a `File` (e.g. a non-hermetic + toolchain's absolute system path); callers should fall back to the + plain tool-path string in that case. - (bool): Whether or not the linker is a direct driver (e.g. `ld`) vs a wrapper (e.g. `gcc`). - (sequence): A flattened command line flags for given action. - (dict): Environment variables to be set for given action. @@ -398,6 +555,7 @@ def get_linker_and_args(ctx, crate_type, toolchain, cc_toolchain, feature_config user_link_flags = get_cc_user_link_flags(ctx) ld = None + ld_file = None ld_is_direct_driver = False link_args = [] link_env = {} @@ -444,8 +602,20 @@ def get_linker_and_args(ctx, crate_type, toolchain, cc_toolchain, feature_config ) ld_is_direct_driver = False + # `cc_common.get_tool_for_action()` returns a plain string. Recover the + # backing `File`, when there is one, so `--codegen=linker=` can be + # passed through `Args` as a `File` and stay eligible for Bazel's path + # mapping (see `_resolve_tool_file`). A miss (the common case -- a + # non-hermetic toolchain's absolute system path) leaves `ld_file` as + # `None` and the caller falls back to the plain string, exactly as + # before this change. + matched_file, suffix = _resolve_tool_file(ld, cc_toolchain.all_files) + if matched_file: + ld_file = matched_file if suffix == None else struct(file = matched_file, suffix = suffix) + if not ld or toolchain.linker_preference == "rust": - ld = toolchain.linker.path + ld_file = toolchain.linker + ld = ld_file.path ld_is_direct_driver = toolchain.linker_type == "direct" # Make sure we include RPATHs for Rust ABI dylibs even when no cc_toolchain. @@ -516,7 +686,7 @@ def get_linker_and_args(ctx, crate_type, toolchain, cc_toolchain, feature_config for element in link_env["LIB"].split(";") ]) - return ld, ld_is_direct_driver, link_args, link_env + return ld, ld_file, ld_is_direct_driver, link_args, link_env def symlink_for_ambiguous_lib(actions, toolchain, crate_info, lib): """Constructs a disambiguating symlink for a library dependency. @@ -1191,6 +1361,15 @@ def construct_arguments( # greater than 1) is used. map_flag = _remove_codegen_units if _will_emit_object_file(emit) else None + # Files resolved by `_resolve_tool_file` for path-mapping-aware rendering + # (the linker and any --sysroot= it needed): these reach the action's + # command line through `Args`, which does not by itself register a File + # as an action input. `collect_inputs`'s own `cc_toolchain.linker_files()` + # has not proven a reliable source for these specifically (see #4250), + # so they are tracked explicitly and merged into `compile_inputs` by the + # caller instead. + extra_action_inputs = [] + # Rustc arguments rustc_flags = ctx.actions.args() rustc_flags.set_param_file_format("multiline") @@ -1349,7 +1528,7 @@ def construct_arguments( else: rpaths = depset() - ld, ld_is_direct_driver, link_args, link_env = get_linker_and_args( + ld, ld_file, ld_is_direct_driver, link_args, link_env = get_linker_and_args( ctx, crate_info.type, toolchain, @@ -1360,11 +1539,46 @@ def construct_arguments( ) env.update(link_env) - rustc_flags.add(ld, format = "--codegen=linker=%s") + needs_linker_files_fallback = False + if ld_file == None: + rustc_flags.add(ld, format = "--codegen=linker=%s") + elif type(ld_file) == "File": + rustc_flags.add(ld_file, format = "--codegen=linker=%s") + extra_action_inputs.append(ld_file) + needs_linker_files_fallback = ld_file.path != ld + else: + rustc_flags.add_all([ld_file], map_each = _tool_file_path, format_each = "--codegen=linker=%s") + extra_action_inputs.append(ld_file.file) + needs_linker_files_fallback = (ld_file.file.path + "/" + ld_file.suffix) != ld # Split link args into individual "--codegen=link-arg=" flags to handle nested spaces. # Additional context: https://github.com/rust-lang/rust/pull/36574 - rustc_flags.add_all(link_args, format_each = "--codegen=link-arg=%s") + wrapped_link_args, sysroot_needs_fallback = _wrap_sysroot_link_args(link_args, cc_toolchain) + for wrapped_arg in wrapped_link_args: + if type(wrapped_arg) == "struct": + extra_action_inputs.append(wrapped_arg.file) + rustc_flags.add_all( + wrapped_link_args, + map_each = _map_each_link_arg, + format_each = "--codegen=link-arg=%s", + ) + + # Belt-and-suspenders: `collect_inputs`' own + # `cc_toolchain.linker_files()` inclusion has not proven + # reliable on every supported Bazel version (see #4250's PR + # history). An *exact* match (the common case for a hermetic + # toolchain, and the only case exercised on most Bazel versions) + # has proven sufficient on its own; the extra flatten here is + # for the specific case a resolution had to fall back to a + # config-agnostic or directory-relative match instead -- + # `ld_file`/a --sysroot= File's own path landing on something + # other than the original string is exactly that signal. + if cc_toolchain and (needs_linker_files_fallback or sysroot_needs_fallback): + # Same version-compat access pattern as collect_inputs() above. + if hasattr(cc_toolchain, "_linker_files"): + extra_action_inputs.extend(cc_toolchain._linker_files.to_list()) + else: + extra_action_inputs.extend(cc_toolchain.linker_files().to_list()) if remap_path_prefix != None and _should_add_oso_prefix( toolchain, @@ -1587,6 +1801,7 @@ def construct_arguments( extra_rustc_flags = rust_flags_args, supports_path_mapping = not target_has_location_expansion, all = all_args, + extra_action_inputs = extra_action_inputs, ) return args, env @@ -1955,11 +2170,18 @@ def rustc_compile_action( if use_split_debuginfo: action_outputs.append(dwo_outputs) # buildifier: disable=uninitialized + # `args`/`args_metadata` (built by construct_arguments) may reference a + # resolved linker or --sysroot= File that `compile_inputs` (built earlier + # by collect_inputs, from the same cc_toolchain) does not reliably carry + # -- see the extra_action_inputs comment in construct_arguments. Merge + # them in here rather than depend on that path. + compile_inputs_for_action = depset(args.extra_action_inputs, transitive = [compile_inputs]) if args.extra_action_inputs else compile_inputs + if ctx.executable._process_wrapper: # Run as normal ctx.actions.run( executable = ctx.executable._process_wrapper, - inputs = compile_inputs, + inputs = compile_inputs_for_action, outputs = action_outputs, env = env, arguments = args.all, @@ -1976,9 +2198,10 @@ def rustc_compile_action( execution_requirements = {"supports-path-mapping": ""} if args.supports_path_mapping else None, ) if args_metadata: + compile_inputs_for_metadata = depset(args_metadata.extra_action_inputs, transitive = [compile_inputs]) if args_metadata.extra_action_inputs else compile_inputs ctx.actions.run( executable = ctx.executable._process_wrapper, - inputs = compile_inputs, + inputs = compile_inputs_for_metadata, outputs = [build_metadata] + [x for x in [rustc_rmeta_output] if x], env = env, arguments = args_metadata.all, @@ -1999,7 +2222,7 @@ def rustc_compile_action( fail("build_env_files, build_flags_files, stamp, build_metadata are not supported when building without process_wrapper") ctx.actions.run( executable = ctx.executable._bootstrap_process_wrapper, - inputs = compile_inputs, + inputs = compile_inputs_for_action, outputs = action_outputs, env = env, arguments = [args.rustc_path, args.rustc_flags], diff --git a/test/unit/path_mapped_linker/BUILD.bazel b/test/unit/path_mapped_linker/BUILD.bazel new file mode 100644 index 0000000000..03da494033 --- /dev/null +++ b/test/unit/path_mapped_linker/BUILD.bazel @@ -0,0 +1,90 @@ +load("@rules_cc//cc:defs.bzl", "cc_toolchain") +load("//rust:defs.bzl", "rust_binary") +load(":path_mapped_linker_test.bzl", "fake_cc_config", "linker_is_path_mappable_test") + +genrule( + name = "gen_fake_linker", + srcs = ["fake_gcc_template.sh"], + outs = ["fake_gcc"], + cmd = "cp $(location :fake_gcc_template.sh) $@ && chmod +x $@", + executable = True, +) + +genrule( + name = "gen_fake_sysroot", + outs = ["fake_sysroot"], + cmd = "echo 'not a real sysroot' > $@", +) + +filegroup( + name = "fake_linker_files", + srcs = [ + ":gen_fake_linker", + ":gen_fake_sysroot", + ], +) + +filegroup( + name = "empty", + srcs = [], +) + +fake_cc_config( + name = "fake_cc_toolchain_config", + linker = ":gen_fake_linker", + sysroot = ":gen_fake_sysroot", +) + +cc_toolchain( + name = "fake_cc_toolchain_impl", + all_files = ":fake_linker_files", + compiler_files = ":fake_linker_files", + linker_files = ":fake_linker_files", + dwp_files = ":empty", + objcopy_files = ":empty", + strip_files = ":empty", + supports_param_files = 0, + toolchain_config = ":fake_cc_toolchain_config", + toolchain_identifier = "path-mapped-fake-cc", +) + +# Scopes `fake_cc_toolchain` so it can only ever be selected for this +# package's own test target, never for an unrelated build elsewhere in the +# graph (e.g. util/process_wrapper, a Rust tool this fake linker would +# silently "link" -- it creates its declared output but never actually +# links anything). `--extra_toolchains` alone makes a toolchain a candidate +# everywhere a matching, unconstrained toolchain_type is requested; +# target_compatible_with -- satisfied only by :fake_platform, which nothing +# else builds under -- is what actually excludes it from every other +# target's toolchain resolution. +constraint_setting(name = "fake_toolchain_marker_setting") + +constraint_value( + name = "fake_toolchain_marker", + constraint_setting = ":fake_toolchain_marker_setting", +) + +platform( + name = "fake_platform", + constraint_values = [":fake_toolchain_marker"], + parents = ["@platforms//host:host"], +) + +toolchain( + name = "fake_cc_toolchain", + target_compatible_with = [":fake_toolchain_marker"], + toolchain = ":fake_cc_toolchain_impl", + toolchain_type = "@bazel_tools//tools/cpp:toolchain_type", +) + +rust_binary( + name = "bin", + edition = "2018", + srcs = ["main.rs"], + tags = ["manual", "nobuild"], +) + +linker_is_path_mappable_test( + name = "path_mapped_linker_test", + target_under_test = ":bin", +) diff --git a/test/unit/path_mapped_linker/fake_gcc_template.sh b/test/unit/path_mapped_linker/fake_gcc_template.sh new file mode 100644 index 0000000000..4f67df6509 --- /dev/null +++ b/test/unit/path_mapped_linker/fake_gcc_template.sh @@ -0,0 +1,27 @@ +#!/bin/sh +# Fake "linker" for this directory's regression test. A real link is never +# meant to succeed here -- the point is only to check the flags rustc built +# for --codegen=linker= and --sysroot=. `bazel test` on the wrapping +# analysistest never executes this (it only inspects the analysis-time +# action graph), but `bazel coverage` does actually build and run the +# target under test's own actions, so this still has to produce *a* file +# at whichever output path it's given, or Bazel's "output was not created" +# check fails the build outright. +out="" +prev="" +for arg in "$@"; do + case "$prev" in + -o) out="$arg" ;; + esac + case "$arg" in + /OUT:*) out="${arg#/OUT:}" ;; + esac + prev="$arg" +done +if [ -n "$out" ]; then + # A shell builtin, not the external `touch`: the sandbox PATH a linker + # is invoked under is minimal and doesn't reliably have it (observed as + # "touch: not found" from this script's own stderr in CI). + : > "$out" +fi +exit 0 diff --git a/test/unit/path_mapped_linker/main.rs b/test/unit/path_mapped_linker/main.rs new file mode 100644 index 0000000000..f328e4d9d0 --- /dev/null +++ b/test/unit/path_mapped_linker/main.rs @@ -0,0 +1 @@ +fn main() {} diff --git a/test/unit/path_mapped_linker/path_mapped_linker_test.bzl b/test/unit/path_mapped_linker/path_mapped_linker_test.bzl new file mode 100644 index 0000000000..0aa6f87261 --- /dev/null +++ b/test/unit/path_mapped_linker/path_mapped_linker_test.bzl @@ -0,0 +1,117 @@ +"""Regression test for #4250: the C++-toolchain-provided linker must reach +rustc through `Args` as a `File`, not a plain string, so Bazel's path +mapping (`--experimental_output_paths=strip`) can rewrite it. + +The bug only manifests for a hermetic/generated C++ toolchain -- one whose +linker tool is itself a Bazel artifact under `bazel-out/...` rather than an +absolute system path like `/usr/bin/gcc`. This test builds a minimal fake +cc_toolchain whose "linker" is a genrule output for exactly that reason. +""" + +load("@bazel_skylib//lib:unittest.bzl", "analysistest", "asserts") +load("@bazel_tools//tools/build_defs/cc:action_names.bzl", "CPP_LINK_EXECUTABLE_ACTION_NAME") +load("@rules_cc//cc:cc_toolchain_config_lib.bzl", "action_config", "flag_group", "flag_set", "tool", "tool_path") +load("@rules_cc//cc/common:cc_common.bzl", "cc_common") +load("@rules_cc//cc/toolchains:cc_toolchain_config_info.bzl", "CcToolchainConfigInfo") +load( + "//test/unit:common.bzl", + "assert_argv_contains_prefix_suffix", +) + +def _fake_cc_config_impl(ctx): + return cc_common.create_cc_toolchain_config_info( + ctx = ctx, + toolchain_identifier = "path-mapped-fake-cc", + host_system_name = "unknown", + target_system_name = "unknown", + target_cpu = "unknown", + target_libc = "unknown", + compiler = "unknown", + abi_version = "unknown", + abi_libc_version = "unknown", + # `tool_path`'s `path` is resolved relative to the cc_toolchain's own + # package (see rules_cc's `get_relative_path`/`_compute_tool_paths`), + # not as an already-exec-root-relative string -- it cannot reference a + # generated artifact whose real location is under `bazel-out//...`. + # Every legacy name must still be declared for `cc_toolchain` to accept + # this config at all, so these are unused, never-invoked placeholders. + tool_paths = [ + tool_path(name = name, path = "/usr/bin/false") + for name in ("ar", "cpp", "gcc", "gcov", "ld", "nm", "objcopy", "objdump", "strip", "dwp", "llvm-profdata") + ], + # `tool(tool = )`, by contrast, is exactly the mechanism a real + # hermetic toolchain uses to point an action at a generated artifact: + # `cc_common.get_tool_for_action()` then returns that File's own exec + # path directly, config-dependent prefix and all -- reproducing what + # #4250 reports, and letting `rustc.bzl`'s `_resolve_tool_file` find it. + # + # The `--sysroot=` flag_group is the other half of #4250: it is how a + # real toolchain config emits a sysroot path, and + # `cc_common.get_memory_inefficient_command_line()` returns it as a + # single already-formatted string in `link_args` -- exactly what + # `_wrap_sysroot_link_args` has to recognise and re-associate with + # `ctx.file.sysroot`. + action_configs = [ + action_config( + action_name = CPP_LINK_EXECUTABLE_ACTION_NAME, + enabled = True, + tools = [tool(tool = ctx.file.linker)], + flag_sets = [ + # An action_config's own flag_sets apply implicitly to its + # action_name; specifying `actions` here is rejected. + flag_set( + flag_groups = [flag_group(flags = ["--sysroot=" + ctx.file.sysroot.path])], + ), + ], + ), + ], + ) + +fake_cc_config = rule( + implementation = _fake_cc_config_impl, + attrs = { + "linker": attr.label(allow_single_file = True, mandatory = True), + "sysroot": attr.label(allow_single_file = True, mandatory = True), + }, + provides = [CcToolchainConfigInfo], +) + +def _linker_is_path_mappable_test_impl(ctx): + env = analysistest.begin(ctx) + tut = analysistest.target_under_test(env) + actions = [a for a in tut.actions if a.mnemonic == "Rustc"] + asserts.true( + env, + len(actions) == 1, + "expected exactly one Rustc action, got mnemonics: {}".format([a.mnemonic for a in tut.actions]), + ) + if actions: + # `analysis_test_transition` (which `config_settings` below drives) + # refuses to set --experimental_* / --incompatible_* options, so this + # test cannot force --experimental_output_paths=strip on itself -- + # only a whole invocation can (see the repo's own "Path Mapping + # Linux/RBE/MacOS" CI jobs, which run this same test suite with the + # flag build-wide). Checking only the suffix keeps this test valid + # either way: run normally it passes trivially; run under + # --experimental_output_paths=strip, a --codegen=linker= built from a + # `File` is rewritten to the literal `bazel-out/cfg/...` prefix, + # while the pre-#4250 plain-string form keeps the real, unmapped + # configuration segment instead -- verified manually both ways. + assert_argv_contains_prefix_suffix(env, actions[0], "--codegen=linker=", "/fake_gcc") + + # Same bug, the other half: a --sysroot= link arg pointing at a + # generated artifact. + assert_argv_contains_prefix_suffix(env, actions[0], "--codegen=link-arg=--sysroot=", "/fake_sysroot") + return analysistest.end(env) + +linker_is_path_mappable_test = analysistest.make( + _linker_is_path_mappable_test_impl, + config_settings = { + str(Label("//rust/settings:toolchain_linker_preference")): "cc", + "//command_line_option:extra_toolchains": [str(Label("//test/unit/path_mapped_linker:fake_cc_toolchain"))], + # Confines the fake toolchain to this test's own configuration -- + # see the target_compatible_with comment on :fake_cc_toolchain in + # BUILD.bazel for why this is necessary, not just belt-and-braces. + "//command_line_option:platforms": [str(Label("//test/unit/path_mapped_linker:fake_platform"))], + }, +)