You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
After some debugging, I believe I have found some kind of bug or issue with how User-Agent headers are handled by httpx as provided within the Python Worker runtime.
The following snippet sets up a simple worker that makes a few requests, and outputs the result.
class Default(WorkerEntrypoint):
async def fetch(self, request):
# A manually constructed request for reference
req = Request.new("https://httpbin.org/headers")
req.headers.set("User-Agent", "some custom user-agent", "Custom-Header")
req.headers.set("Custom-Header", "another value")
result = await fetch(req)
print(await result.json())
# An httpx driven request (which does not function as expected)
print(
(
await httpx.AsyncClient().get(
"https://httpbin.org/headers",
headers={
"User-Agent": "some custom user-agent",
"Custom-Header": "another value",
},
)
).json()
)
Aside from the few extra headers, the request through httpx does not have the custom User-Agent. It appears to have been uniquely dropped at some point, since the other Custom-Header survives.
I have exhausted most other debugging options, and am left thinking that this must be either an intentional exemption in the runtime (preventing User-Agent from being overridden, but only sometimes?) or a bug internally with how httpx is bridged to Python Workers.
When htps is run in a local script outside of the Python Worker runtime, the correct User-Agent header is attached.
Thanks for the report. We can patch httpx to fix that, but httpx is currently not maintained and we recommend replacing it with httpx2, which is better maintained by Pydantic team.
I wonder if you are using httpx directly, or using it as a dependency of another package. If former is the case, could you try to replace it with httpx2?
I am using httpx directly, so I could swap to httpx2 - however, the same issue occurs with httpx2 as well.
Whichever is easiest to fix upstream, I'm happy to use, but it does seem like both versions have this same problem.
I will say that 90% of the Python Workers docs I have seen refer to httpx if they refer to any third party HTTP client, so if httpx2 is recommended, it's probably worth auditing and updating all of that documentation to match.
Whichever is easiest to fix upstream, I'm happy to use, but it does seem like both versions have this same problem.
I see. Let us see if we can fix it in upstream httpx2.
I will say that 90% of the Python Workers docs I have seen refer to httpx if they refer to any third party HTTP client, so if httpx2 is recommended, it's probably worth auditing and updating all of that documentation to match.
Yeah, we need to update them all... thanks. There was some drama in httpx/httpx2/httpxyz lately.
This was added intentionally at some point because attempting to set a user agent header in a browser causes a warning saying it's not allowed. But we shouldn't do it in node or workerd.
Activity
[-]π Bug Report β httpx via Cloudflare Python Worker refuses User-Agent headers[/-][+]π Bug Report β httpx via Cloudflare Python Worker drops User-Agent headers[/+]ryanking13 commentedon Sep 7, 2026
Thanks for the report. We can patch httpx to fix that, but httpx is currently not maintained and we recommend replacing it with httpx2, which is better maintained by Pydantic team.
I wonder if you are using httpx directly, or using it as a dependency of another package. If former is the case, could you try to replace it with httpx2?
jGowgiel commentedon Sep 7, 2026
I am using httpx directly, so I could swap to httpx2 - however, the same issue occurs with httpx2 as well.
Whichever is easiest to fix upstream, I'm happy to use, but it does seem like both versions have this same problem.
I will say that 90% of the Python Workers docs I have seen refer to httpx if they refer to any third party HTTP client, so if httpx2 is recommended, it's probably worth auditing and updating all of that documentation to match.
ryanking13 commentedon Sep 7, 2026
I see. Let us see if we can fix it in upstream httpx2.
Yeah, we need to update them all... thanks. There was some drama in httpx/httpx2/httpxyz lately.
hoodmane commentedon Sep 8, 2026
This was added intentionally at some point because attempting to set a user agent header in a browser causes a warning saying it's not allowed. But we shouldn't do it in node or workerd.