When the URL contains request parameters and the params parameter is set, the request parameters in the URL will disappear unexpectedly. #3621

Activity

1. 2513502304 commented on Aug 5, 2025

   

   2513502304

   on Aug 5, 2025

   Author

   More actions

   As shown in the following Python code, my expected output should be to reasonably concatenate the request parameters in params with those in the url. requests handled this part well, but httpx gave the following unexpected output:

   import requests
   import httpx
   
   url = 'https://safebooru.org/index.php?page=post&s=list'
   params = {
       'pid': 0,
       'tags': 'k-on!',
   }
   
   response = requests.get(url=url, params=params)
   print(response.request.url)  # As expected: https://safebooru.org/index.php?page=post&s=list&pid=0&tags=k-on%21
   
   response = httpx.get(url=url, params=params)
   print(response.request.url)  # Unexpectedly, the request parameters in the URL were overridden by params: https://safebooru.org/index.php?pid=0&tags=k-on%21
   

   My httpx version is 0.28.1

   👍React with 👍2Reacted by James Braza and mskarlin
2. techouse commented on Aug 28, 2025

   

   techouse

   on Aug 28, 2025

   Last edited by techouse

   More actions

   If you want to avoid params getting dropped/overwritten when a URL already contains a query, you can pre-merge the query yourself with qs_codec (my Python port of the Node.js qs library) and hand httpx a fully-formed URL. This also gives you proper nested param handling (a[b][0]=...) and configurable encoding (spaces as %20 vs +, etc).

   Safely merge existing URL query with new params

   import httpx
   from urllib.parse import urlsplit, urlunsplit
   import qs_codec as qs
   
   base_url = "https://api.example.com/search?a=b&c=d"  # already has query
   new_params = {
       "c": "D",               # will co-exist or override per your choice below
       "e": "f",
       "tags": ["x", "y"],     # lists -> multiple entries or bracketed, as configured
   }
   
   parts = urlsplit(base_url)
   
   # Decode existing query string into a nested dict (robust parsing, preserves duplicates)
   existing = query_string.decode(parts.query)
   
   # Merge how you want (examples):
   # a) combine duplicates into a list
   for k, v in new_params.items():
       if k in existing:
           # promote to list and append
           existing[k] = ([existing[k]] if not isinstance(existing[k], list) else existing[k]) + (v if isinstance(v, list) else [v])
       else:
           existing[k] = v
   
   # b) OR choose "last-wins" by just updating:
   # existing.update(new_params)
   
   # Re-encode exactly once (RFC 3986 = spaces as %20; switch to RFC1738 for '+')
   query_string = qs.encode(existing, qs.EncodeOptions(format=qs.Format.RFC3986))
   
   final_url = urlunsplit((parts.scheme, parts.netloc, parts.path, query_string, parts.fragment))
   resp = httpx.get(final_url)

   Nested query parameters (qs-style)

   If the API expects bracketed or indexed keys (a common pain point), qs_codec handles it:

   # Build nested structure directly in Python
   payload = {
       "search": {"term": "billing", "filters": [{"field": "status", "op": "eq", "value": "open"}]},
       "page": 2
   }
   
   # Encodes to: search[term]=billing&search[filters][0][field]=status&...&page=2
   query_string = qs.encode(payload)  # percent-encoded by default
   url = f"https://api.example.com/tickets?{query_string}"
   resp = httpx.get(url)

   Encoding edge cases (spaces, %, etc.)

   qs_codec lets you pick encoding style and avoids double-encoding:

   • RFC 3986 (default) → space = %20
   • RFC 1738 → space = +

   qs.encode({"q": "foo bar"}, qs.EncodeOptions(format=qs.Format.RFC3986))  # q=foo%20bar
   qs.encode({"q": "foo bar"}, qs.EncodeOptions(format=qs.Format.RFC1738))  # q=foo+bar

   This approach works today without changes to httpx: you bypass the internal params merge (the source of surprises here) and send an already-correct query string.
3. 2513502304

   added a commit that references this issue on Dec 25, 2025

   fix: fix the [issue 3621 of httpx](encode/httpx#3621)

   0e41a37
4. veeceey

   added 3 commits that reference this issue on Feb 8, 2026

   Fix URL parameter merging when both URL and params are provided

   ...

   This issue will close once commit 6e888bc is merged into the 'master' branch.

   6e888bc

   Merge params with existing URL query parameters instead of replacing …

   ...

   This issue will close once commit 58e14dd is merged into the 'master' branch.

   58e14dd

   Merge params with existing URL query parameters instead of replacing …

   ...

   This issue will close once commit 38fe19f is merged into the 'master' branch.

   38fe19f