Missing query params in url when params option is set

In version 0.28.0, query params are never merged.

import httpx

client = httpx.Client()
request = client.build_request(
    "GET", "https://httpbin.org/get?param=default", params={"foo": "bar"}
)
print(request.url)  # This will be `https://httpbin.org/get?foo=bar`

The query string in the request url will be ignored. It seems this is caused by the pr #3364. Params are replaced instead of merged when parsing.

httpx/httpx/_urlparse.py

Line 301 in 15e21e9

query = kwargs.get("query", url_dict["query"])

Related discussion #3428

fix: merge existing params in url, close encode#3433

Params are replaced instead of merged when parsing.

This is exactly the behavior I would expect.
As I see it, the simplification in #3364 introduced a bugfix that we haven't documented in the release.

I know providing both params and query in url is a bad practice. But this can be counterintuitive in some situations like:

import httpx

client = httpx.Client(params={"foo": "bar"})
request = client.build_request(
    "GET", "https://httpbin.org/get?param=default"
)
print(request.url)  # This will be `https://httpbin.org/get?foo=bar`

The params are provided in the client options. This will also replace the query in the url.

May be a warning can be raised when the replaced query is not empty?

httpx/httpx/_urlparse.py

Line 301 in 15e21e9

query = kwargs.get("query", url_dict["query"])

Related issue... Client(params=...) is a not-good idea.

But this can be counterintuitive in some situations like

That is surprising behaviour, yes. I'd consider that a bug.

To the extent that we do currently support Client(params=...) we should probably? be merging those with any params=... passed by the user. Tho it's a bad pattern in the first place that we ideally could have avoided.

Currently, as a workaround, we merge all the params by ourselves before passing to httpx. The code is clearer logically 😂 .

fix(requestor): make params None if it's empty dict to prevent replac…

[back] Upgrade backend dependencies + fix test test_get_log_entities_…

Broke quite some of my code, so at least a warning and a hint on how best to merge parameters should be issued.

Beware, this breaks compatibility with Requests, which does merge the parameters:

It seems like it may be worth restoring the previous behavior for parity with requests? Is there a way we can discourage the "bad pattern" while still merging?

Personally, I believe that the "merge" behaviour is more intuitive for someone who specifies both a url with query parameters and a params dictionary.

I often encounter the case of a GET web service, that can be implemented by many different actors, who will give me a URL string that I will stick in a configuration.
And then, in order to call that web service, I will need to add some parameters that are specific to the call (a user ID, search parameters, etc.)
With the replace behaviour being the default, this means that, in order to handle the case where the original URL contains a query string, I need to do one of these things:

parse the URL and put everything in params so that I do not lose anything (taking in account that httpx query params handling is not necessarily round-tripping for every query string, which can sometimes be important);
parse the URL and use the copy_with_merge_params manually;
detect whether the URL contains a ? and add a string suffix to the URL based on the parameters I want to add;

No matter the solution we recommend for merging, the params argument seems like a trap: params works only if I can be sure that my URL never has a query string - otherwise I need to do the merging dance systematically.

It would be a more honest API, in that case, if the params argument was removed and we said "you need to add the parameters to the URL in order for them to work". However, we should not do that because that would be a significant usability regression.

The new behaviour also makes it more difficult to work with link URLs (e.g. prev, next, last) in http responses – I'm surprised it hasn't been brought up before. Now, extra logic is required to convert the query parameters in each link to a dict and then merge that with any other query parameters that might be required. We aren't short of tools to do that in the stdlib, but adding new logic like this requires extensive testing to ensure correctness because there are several corner cases.

I run into this issue as well. Query params were removed from the url, just because I provided an empty dictionary: params = {}

@tomchristie

As a grateful user (and not in a position to contribute) to httpx, I can understand any decision made by maintainers.
Nevertheless, I'd like to ask @tomchristie to please explain the decision to close this, given that:

it changes the behavior from previous releases and breaks code (as shown in the links to issues in other projects), without a warning in the release notes.
it deviates from requests
some people consider it a practical (albeit maybe special case) feature (along the lines of Special cases aren't special enough to break the rules. Although practicality beats purity. in the Zen of Python).
makes it more difficult to provide link headers commonly used in HATEOAS (see @urschrei 's comment above)
workarounds are somewhat clumsy.

Thanks again for the great httpx and all the effort that go into it.
Martin

So this breaking change isn't going to be reverted? The path forward is to pretend like this was the intended behavior even though there's a clear lineage showing the previous behavior was correct/intentional?

So this breaking change isn't going to be reverted? The path forward is to pretend like this was the intended behavior even though there's a clear lineage showing the previous behavior was correct/intentional?

Stick to httpx==0.27.0 just in case some random requests might be affected since I can't really tell which one would be correct or wrong. This breaking change was introduced in the last digit version up.

[...]please explain[...]

The behavior of params=... matches browser behavior when submitting an HTML <form ...> element.

So this breaking change isn't going to be reverted?

This behavior change isn't going to be reverted.

@lovelydinosaur

I just got bitten by this stupid breaking change...
@lovelydinosaur This is totally not serious to keep this broken new behavior.

It doesn't respect the basic software principle of "Least Surprise".
It's not coherent with Requests behavior while claiming "HTTPX builds on the well-established usability of requests".

Especially changing that without advertising this as a SERIOUS BREAKING CHANGE in the changelog makes of httpx a real piece of crap!

I hope that you will reconsider...

@fviard

@fviard I understand you've experienced frustration...

However, do you realize that your comment doesn't add anything new to the discussion? 🤔
And if httpx was actually "a piece of crap", can't you just go and use something else?
Or build something better?

You're not the first one to have been bitten by this change.
You can always build your own wrapper around HTTPX that implements the behavior of merging params if you actually want that, what's the big deal?

I invite you to learn how to deal with frustration in a more constructive way.

Because nobody owns you a free and open source HTTP library. 🤷

(sorry, I had to react, it's about time people stop treating maintainers as if they were their slaves)

@eliasdorneles

@eliasdorneles Yeah the tone may not be the right one as the one of the package maintainer.

But the point here is not about the tone, it's about introducing breaking changes in such a way that it's really hard to identify.

And if httpx was actually "a piece of crap", can't you just go and use something else?
Or build something better?

Unfortunately, while we are able to switch libs we are using in our own projects, some third part package we depends on are using httpx it under the hood which makes it impossible to simply switch the dependency.
You can say that we can still pin the version we are using to solve the issue but packages may have pinned version 0.28.0 for instance and don't know yet there is breaking changes that has been introduced because it's a breaking change in such a way than it's possible that only the end user of the lib will face it and not the intermediary lib owner.

I invite you to learn how to deal with frustration in a more constructive way.

This is not frustration, this is introducing breaking changes in a minor version without telling anyone nor explaining why such change has happened. It has also been identified as a bug in the change log, so it's flagged as a bugfix even if it has been introduced in this PR with the base branch named refactor-keep-the-code-clean and a commit message: Keep it neat.

From where I'm looking from, it's just a small bug introduction (that's happen) during a refacto from the words of the project maintainer:

This is exactly the behavior I would expect. As I see it, the simplification in #3364 introduced a bugfix that we haven't documented in the release.

As I said, it happens and it's not a big issue. Thousand of bugs are introduced every day on open source project and it's part of the process, we can't be perfect every time. The issue here is not the introduction of the bug but more the maintainer principle position about that.

The good workflow would have been:

introduction of the bug (because yeah, despite what you are saying, this is a bug that has been introduced...)
report of the bug
rollback and release of a patch
AND ONLY THERE discussion about this new behavior (as described in the contributing section of the documentation and as the maintainer already have made some like this one this one)
potential new version with a clear breaking change indication in the change log

First, speaking about a "bugfix introduction" is really weird in that case as it doesn't make sense while it's not identified as a bug (which was not the case) and doesn't even follow the contributing guide of this package.

Second, the attitude of the maintainer is not productive in any way. He has never justified its choice, he has never answer to the community something else than "no, we will not change this behavior" which looks really surprising in a widely used python package.

Third, the community has been discussing in a really good community based approach #3433 (comment)

And finally, even if developers' frustration should never be directed at open source contributors and even more at the project maintainers, it's really naive to think that choices made without communication, without justification, without following the own project contribution guide and even without accepting different point of views (that are the only one that as been argued and justified by the community) will produce something else than frustration.

Maintainers are not gods and they can be wrong. They also have the power to deny they are wrong but it's not making them right, they are just wrong with power.

I apologize if I feel aggressive in my comment, this is not my goal at all and I'm really aware of the condition of open source developers and the stress/aggressiveness that occurs in the community. But I think you don't realize how this breaking change has impacted some people/projects/companies and how frustrating the maintainer behavior can be regarding people that just want to discuss and enhance things such as Jhominal in this comment.

I'm open to discuss more if you want.

@mde-pach

@mde-pach I believe there is a misunderstanding. Read carefully that quote:

the simplification in #3364 introduced a bugfix that we haven't documented in the release

(bold emphasis is mine)

The maintainer considers the behavior change a bugfix -- and not a bug introduction, as you seem to have understood it.
It was documented as such in the changelog.

As I see it, the maintainer clearly stated their position from the very beginning.
There are valid concerns for not wanting to merge automatically: it's confusing.

You may disagree with it, it may be unconventional, but ultimately it's still more subjective than objective.
There is no right and wrong here, it's all about tradeoffs.

People who need merged params can still write a wrapper around HTTPX and merge the params, no?
If you have a dependency that needs that, why not to send a PR to implement that?

it's really naive to think that choices made without communication, without justification, without following the own project contribution guide and even without accepting different point of views (that are the only one that as been argued and justified by the community) will produce something else than frustration.

Isn't it also a bit naive to expect that attacking the project and the people who are working on it and giving it away for free, will somehow improve things?

@eliasdorneles

@eliasdorneles how is this a "bugfix" that was "clearly stated from the very beginning" when this "bug" has been in place for 6+ years and nobody created an issue to "fix" it? This seems to have been one-person decision, not a request from the community, which is where the frustration lies.

If this was their position "from the very beginning" why are they stating here 6 years ago that merging automatically was the intended behavior?

You may disagree with it, it may be unconventional, but ultimately it's still more subjective than objective.
There is no right and wrong here, it's all about tradeoffs.

So why make a subjective, breaking change to something that has been one way for 6+ years? Why call it a "bugfix" if it's subjective?

These aren't personal attacks, they're the questions of users of this library. I don't understand who this software is being maintained for if not the users or why pointing out that a breaking change makes our lives difficult is seen as an attack. Factually this wasn't a bug. It's documented intended behavior that has been in place for 6+ years and follows the way most http libraries work.

Isn't it also a bit naive to expect that attacking the project and the people who are working on it and giving it away for free, will somehow improve things?

What's the effective way for us to improve things so breaking changes aren't pushed forward as "bugfixes" in the future? At this point I've accepted what's done is done, but I would love to feel confident that this won't happen again. None of the response have inspired that confidence, though. If the position is httpx maintainers will push forward any breaking behavior changes they deem necessary to move the library forward, that's within your rights, people would just like to know if that's the case.

@mde-pach

Sorry I was not able to reply earlier, but thanks to @mde-pach and @kyle3s to have clearly stated what I had in mind.

Regarding:

(sorry, I had to react, it's about time people stop treating maintainers as if they were their slaves)

Basically I don't ask the maintainers to "do something" as if they were "slaves", I just request them to not do anything "harmful" to the users out of the blue.
It is a basic Principe of respect to your users when a lot of other components are depending on you, and you kind of "advertised" your stack to reach this point.
If tomorrow the said maintainers decided to introduce a malware or a stupid badly managed rm -rf inside the code as a minor update, I don't think that the point would be the maintainers "exploitation".

That being said, I'm really angry at this because:

If you look at the history, the current behavior was a bug, that was "fixed" with extended discussion in the past.
The "IMPORTANT" behavior change to the current buggy version was done in a very stealthy way. No announcement of breaking change, for a clearly dangerous breaking change.
The matter was closed abruptly with a clear will to not have any discussion or justification for this change.

There is even still a "Pull request" unaddressed from a poor contributor that naively didn't realize that it was not a bug but a feature:
#3618
(Sorry @ferstar, the maintainer doesn't care about proper behavior, ...)

And actually, in theory, you could decide for whatever behavior you want for your library, no problem with that, but then, DON'T LIE pretending to be broadly compatible with "Requests" following on its usability principles.
Don't pretend to be an easy almost drop-in replacement of Requests, if you have the same behavior as "Requests", and then, suddenly and silently, decide one day to break it for everyone.

To this day, this limitation is still not listed there:
https://www.python-httpx.org/compatibility

And for the truth to be said, that issue was the nail of the coffin of migrating everything I could out of httpx.
But, sadly, as said by the others, we don't control which "request" library is used by the libraries we use.

@kyle3s

@kyle3s Okay, I had missed that discussion in #652, I see that indeed the merge behavior was intentionally added in 2019 -- thanks for sharing that.

What's the effective way for us to improve things so breaking changes aren't pushed forward as "bugfixes" in the future?

Thanks for the question, I've been thinking about this, and I believe constructive ways that the community can do is to plead the case arguing with facts (e.g. like @jhominal did in this comment), ask for a clearer vision of HTTPX as library and seek to build a dialogue with the maintainer.

I don't think persecuting the fellow will accomplish anything. 🤷

@fviard I understand that you're angry, and I still don't like your tone. You're reclaiming respect as a user? Well, then treat others with the same respect you'd like to be treated! Think: is this how you treat your co-workers?? Keep in mind that behind every screen there is a human being with feelings.

@lovelydinosaur Hello Kim! 👋

It seems that the crowd have spotted an inconsistency between your past self and your current self -- and some are very very mad..! 😅

Anyway, from what I can see, some users are worried that there might be future behavior changes like this in detriment of backwards compatibility or compatibility with Requests or other libraries.

My understanding is that your vision for the client behavior you're seeking to match is browser's HTML <form ... behavior.

A few questions to help clarify things:

Could you please share a bit more on your vision regarding that?
Would you accept breaking compatibility with Requests in the name of that vision?
What about backwards compatibility?

Thank you!

@kyle3s

I'm the author of PR #3618. I submitted it after being bitten by this silent breaking change in production.

A few points I'd like to add:

Historical context: As @kyle3s pointed out, the merge behavior was intentionally implemented in 2019 by the same maintainer who now calls it a "bug". This inconsistency deserves an explanation.
Industry standard: @jhominal's research shows that all 11 libraries with similar API patterns implement merge behavior. httpx is now the only outlier.
On @fviard's frustration: I understand why some feel his tone was harsh. But let's be fair — when you get bitten by a silent breaking change in production, discover there's an ignored PR fixing it, and see the maintainer dismiss community feedback with "this won't be reverted" without addressing the historical inconsistency... frustration is a natural response. The tone police shouldn't distract from the valid technical arguments.
Practical workaround: For those affected, here's a wrapper I use in my projects:

from urllib.parse import urlparse, parse_qs, urlencode, urlunparse

def merge_url_params(url: str, params: dict) -> str:
    """Merge params with existing query parameters in URL."""
    parsed = urlparse(url)
    existing = {k: v[0] for k, v in parse_qs(parsed.query, keep_blank_values=True).items()}
    existing.update({k: str(v) for k, v in params.items()})
    return urlunparse(parsed._replace(query=urlencode(existing)))


# Usage: merge params before passing to httpx
url = merge_url_params("https://api.example.com?token=abc", {"page": "1"})
response = await client.get(url)
# Result: https://api.example.com?token=abc&page=1

I've accepted that my PR likely won't be merged. But I do think the community deserves a clear answer: was the original merge behavior (that existed for 5+ years) a bug or a feature? You can't have it both ways.

@ferstar

@ferstar thanks for giving a good example of constructive behavior!

About your 3rd point: of course frustration is the natural response, I believe everybody who participated in the thread experienced it, myself included. Yet nobody attacked verbally the project and the people maintaining it, or at least not publicly. I don't care about policing tone, but it's important for me that certain limits are respected, which is why I reacted.

Thanks for the discussions, I'll take a break from this thread now. 😅

Merry Xmas everyone!

@lovelydinosaur

I would just like to come back to this discussion to note two things:

I am reassured to see that I am not the only one to be bothered by the way that this change was handled (I am fine with the way that it was implemented, and can accept that no one had realized that it would break existing usages - but I do not agree with the way that it was promoted and sold as a “surprise bugfix” when it was released);
having read the issues that are related to the way that parameter merge was buggy (though not recently), I believe that all these bugs came from the way that parameter merge was implemented in httpx, by first reparsing the query string in the URL. I believe it would be possible to have both parameter merging and avoid the bugs that were present in the old versions of httpx, by merging the query string in a purely additive manner (i.e. without reparsing the query string given in the url). I would be happy to write a PR for such a change, in case there is any sign from @lovelydinosaur that a behaviour change would be considered.

@jhominal

@jhominal Your analysis is spot on. The "purely additive" approach sounds like a reasonable middle ground — it addresses the maintainer's concerns about the old implementation's bugs while restoring the merge behavior that users expect.

If you do submit a PR, I'd be happy to close mine (#3618) in favor of yours, or help in any way I can. A solution backed by your thorough research would carry more weight.

@eliasdorneles Thanks, and Merry Xmas to you too! 🎄

And to @lovelydinosaur — despite our disagreement on this issue, I genuinely appreciate your years of dedication to httpx. Maintaining a widely-used open source project is no easy task. Thank you for your work.

Navigation Menu

Repository navigation

Missing query params in url when params option is set #3433

3 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Missing query params in url when params option is set #3433

Description

Activity

lovelydinosaur commented on Dec 3, 2024

yanyongyu commented on Dec 3, 2024

lovelydinosaur commented on Dec 3, 2024

yanyongyu commented on Dec 3, 2024

mcgfeller commented on Dec 29, 2024

eliasdorneles commented on Jan 16, 2025

zanieb commented on Jan 26, 2025

jhominal commented on Jan 26, 2025

urschrei commented on Jan 26, 2025

danieldotnl commented on Feb 7, 2025

mcgfeller commented on Feb 9, 2025

3 remaining items

kyle3s commented on May 23, 2025

boltomli commented on May 26, 2025

lovelydinosaur commented on May 26, 2025

fviard commented on Dec 9, 2025

eliasdorneles commented on Dec 9, 2025

mde-pach commented on Dec 22, 2025

eliasdorneles commented on Dec 23, 2025

kyle3s commented on Dec 23, 2025

fviard commented on Dec 23, 2025

eliasdorneles commented on Dec 23, 2025

ferstar commented on Dec 24, 2025

eliasdorneles commented on Dec 24, 2025

jhominal commented on Dec 24, 2025

ferstar commented on Dec 24, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions