withResponseHistory: recv: invalid argument (Bad file descriptor)

[dzhus]

I'm experiencing the following issue with http-client-0.4.24, on Mac OS X 10.10.5 and Ubuntu 14.04.

import Prelude
import Network.HTTP.Client

followUrl' :: Manager
           -> String
           -> IO String
followUrl' mgr url = do
    req <- parseUrl url
    withResponseHistory req mgr $ \hr ->
        return $ show $ getUri $ hrFinalRequest hr

This fails with Bad file descriptor error for URLs which produce 404s:

λ> newManager defaultManagerSettings >>= (\m -> parseUrl "http://www.nytimes.com/2015/12/11/" >>= \r -> httpLbs r m)
*** Exception: StatusCodeException (Status {statusCode = 404, statusMessage = "Not Found"}) [("Server","Apache"),("X-App-Name","error"),...
λ> newManager defaultManagerSettings >>= (`followUrl'` "http://www.nytimes.com/2015/12/11/")
*** Exception: recv: invalid argument (Bad file descriptor)

Any ideas?

[snoyberg]

Some initial information:

1. I can reproduce the problem
2. This doesn't seem to be a general problem with 404 responses, but rather specific to nytimes.com

[snoyberg]

I've pushed a commit to master which should address the problem, can you give it a shot?

[dzhus]

Thank you Michael, I think you've fixed it – the bug does not occur anymore.

Another URLs I've seen it happen with:

• http://dw.com/p/1HLgB?tw
• http://eonli.ne/1NProGo
• http://martha.ms/6011BTtHJ
• http://ss1.us/a/J09vNDCu
• http://uk.reuters.com/article/idUKKBN0TT18820151210

[snoyberg]

Cool, thanks for confirming. Uploaded to Hackage and closing.

[Yuras]

Just saw it in the changelog on hackage. I think here we have bigger issue: we are trying to read from already closed socket, which is unsafe.

According to docs, any operation on closed socket will fail, but it is not always the case. In this particular case, recv may return data from an arbitrary connection which reused the same FD. It is horrible and almost impossible to debug. You may argue that it is unlikely, but I saw it in practice.

The solution could be to check status before reading from or writing to socket.

[snoyberg]

This is news to me, I was under the impression that the calls in network would prevent this via their usage of MVar, but I suppose not. Would you be able to either:

1. Open up a new issue about this, or (better yet)
2. Send a PR to do the checking that you describe?