[question] `replace_tool_requires` is confusing

[timpatt]

What is your question?

When (host) conflicts arise with a "requires" requirement, we can use the "replace_requires" to fix them. Conflicts in "tool_requires" cannot be fixed using "replace_tool_requires", in the common case, though...

Some more detail:
I have a conanfile that has a dependency on Qt. In the particular version of the Qt recipe used, there is a conflict with libffi. To solve the conflict, I added the following to my host.profile file:

[replace_requires]
# Resolve conflict in qt 6.8.3 transitive dependencies between glib/2.85.3 (currently resolves to libffi/3.4.8) and wayland 1.22.0 (locked to libffi/3.4.4)
libffi/3.4.4: libffi/3.4.8

That solved the problem.

Now I add qt as a tool requires to my conanfile (this is contrived but required on the Windows build). Of course, the same conflict arose again, but this time in the build profile.

The first issue I had was that the error message didn't indicate that the build profile had the issue - I assumed it was the host profile and took me a while to work it out: "ERROR: Version conflict: Conflict between libffi/3.4.4 and libffi/3.4.8 in the graph. Conflict originates from wayland/1.22.0".

Following the pattern used in the host profile, the solution to the conflict should be obvious; add the following lines to my host.profile file:

[replace_tool_requires]
libffi/3.4.4: libffi/3.4.8

That seems reasonable to me, but it didn't work and was just ignored. I added the same lines to a "build.profile", but that didn't work either. What does work is to add replace_requires to the "build.profile". This makes sense, but I just spent the last 2 hours trying to work it out...

Repro:
conan lock create . -pr:h host.txt -pr:b build.txt

conanfile.py
host.txt
build.txt

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[timpatt]

Maybe this should've been a bug; the solution could probably just be a little extra documentation on the replace_tool_requires attribute. And an indication of host/build on the conflict output.

[memsharded]

Hi @timpatt

Thanks for the feedback.

We already included in the docs in https://docs.conan.io/2/reference/config_files/profiles.html#replace-requires, an "important" highlighted box that summarizes the expectations:

This section should be added to the profile whose context is the one that requires the tool, i.e.,
if the tool is required in the host context, then it should be added to the host profile, so that 
the requirement itself can be replaced.

Maybe this message should be further extended and clarified?

Showing a message when a conflict occurs is way more challenging, because this replacements didn't happen at all, but that is not always and issue, in fact it is very common as profiles are applied to different packages, many of them will not really use the replacement, and that is not related to possible conflicts.

[timpatt]

Heya @memsharded,

I did see that highlighted box while trying to decipher what was happening, but it didn't really help me understand the situation. The fact that a "requires" can be overridden by "replace_requires", but a "tool_requires" in the default case has no (?) relationship to "replace_tool_requires" is confusing and might suggest a lack of understanding on my part, or maybe there's a naming "smell" here. I suspect another call-out or addition to the existing one identifying the common use-case - and maybe giving an example - would help.

e.g. (very rough and probably wrong in many ways!)

By default, tool_requires are placed into the build context. Replacing "requires" for these tools should be done using the replace_requires section applied using a build profile. replace_tool_requires when used in a host profile is intended for replacing tools that have been forced into the host context using force_host_context.

The "replaced requires" section that is output from conan is a suitable for debugging the "replace_requires" functionality; my issue was with the final "conflict error" message which didn't indicate which context it was associated with; would it be possible to add a "build-context" or "host-context" identifier somewhere in the message?

Finally- I tried to use a "patch-version-only" wildcard in replace_requires and noticed that it didn't work. e.g. libffi/3.4.*: libffi/3.4.8. Is this intentional? I'm keen to limit the scope of the replacements, but only "fine-grained" (i.e. match exact version) or "broad" (i.e. "match every version") seem to be supported at the moment. I know that the use of version ranges are limited to a very small subset of recipes on conancenter ATM so being able to "replace requires" at the minor/patch/whatever level would be quite helpful in reducing the work required to "clean up" a graph.

[memsharded]

but a "tool_requires" in the default case has no (?) relationship to "replace_tool_requires" is confusing and might suggest a lack of understanding on my part, or maybe there's a naming "smell" here.

But a replace_tool_requires can replace tool_requires() from recipes, in exactly the same way.
The difference is the context.
If you put a replace_tool_requires in the "host" profile it will replace the tool_requires() that it finds in the packages in the "host" context. For example if a zlib recipe has a tool_requires("cmake/xxx")

That scenario is different if what you want to replace are transitive dependencies of the tools that live inside tool-requires packages. Those live in the "build" context and are only affected by definition in the "build" profile. Both replace_requires and replace_tool_requires will affect in the same way, replacing requires with replace_requires and tool_requires with replace_tool_requires.

I suspect another call-out or addition to the existing one identifying the common use-case - and maybe giving an example - would help.

Yes, I can see that, lets try to add some further clarifications there in the docs.

Finally- I tried to use a "patch-version-only" wildcard in replace_requires and noticed that it didn't work. e.g. libffi/3.4.*: libffi/3.4.8. Is this intentional? I'm keen to limit the scope of the replacements, but only "fine-grained" (i.e. match exact version) or "broad" (i.e. "match every version") seem to be supported at the moment. I know that the use of version ranges are limited to a very small subset of recipes on conancenter ATM so being able to "replace requires" at the minor/patch/whatever level would be quite helpful in reducing the work required to "clean up" a graph.

The patch version only that you are trying seems that it should be a version range? Like libffi/[3.4.*] instead?