Cabal install fails when ** wildcards are used

[mrkkrp]

See NixOS/nixpkgs#64173 (comment)

[quasicomputational]

I can't replicate this locally with just v2-install (i.e., no Nix). v1-install is chugging away but it'll be a minute because it's installing about five million things. (And, it worked, so I can't reproduce it at all, it seems.)

-v3 output may be helpful, if someone who can replicate the failure (maybe @cdepillabout?) could do that.

[cdepillabout]

@quasicomputational The full cabal v2-install -v3output was quite verbose, so I put it in a gist:

https://gist.github.com/cdepillabout/9a2ca42fcf78fa9462b63ea3c4a8f533#file-cabal-install-output-with-error-txt

This is on NixOS, getting GHC and cabal from Nix (using the shell.nix in ormolu's repo).

This is using the following cabal:

$ cabal --version
cabal-install version 2.4.1.0
compiled using version 2.4.1.0 of the Cabal library 

[quasicomputational]

Thanks!

Here's what look like the critical lines:

Installing ./data/printer/ssig-single.hs to
/home/illabout/.cabal/store/ghc-8.6.4/incoming/new-15111/home/illabout/.cabal/store/ghc-8.6.4/ormolu-0.0.1.0-92a9154ca931fd51a93dff32cdbea6dc0ddb6dbdc5d057c74b04741d826ca4f7/share/data/printer/ssig-single.hs
/home/illabout/.cabal/store/ghc-8.6.4/incoming/new-15111/home/illabout/.cabal/store/ghc-8.6.4/ormolu-0.0.1.0-92a9154ca931fd51a93dff32cdbea6dc0ddb6dbdc5d057c74b04741d826ca4f7/share/data/printer: copyFile: does not exist (No such file or directory)
CallStack (from HasCallStack):
die', called at ./Distribution/Client/ProjectOrchestration.hs:1002:55 in main:Distribution.Client.ProjectOrchestration

To me, that looks like copyFile is trying to act on a directory and failing, which sounds sort of plausible if somehow the globbing code is broken. That line is very likely being logged from installOrdinaryFile, which has a few call sites, but a reasonable guess is that Distribution.Simple.Install is one of the culprits, and in particular a function called installDataFiles stands out as a candidate.

Unfortunately, the code there looks reasonable to me and just eyeballing it doesn't reveal an obvious bug.

I don't understand why this would only be occurring in a Nix environment - when I try it locally, I don't get that second Installing line only naming the directory, but only ones naming files.

Just to confirm where the bug is, could you run this script (with cabal new-run test.hs) in the ormolu directory and put the output in a gist, please?

#!/usr/bin/env cabal
{- cabal:
build-depends: base ^>= 4.12
             , Cabal ^>= 2.4
-}
import Data.Foldable
import Distribution.Simple.Glob
import Distribution.Verbosity
import Distribution.Version

main =
  matchDirFileGlob normal (mkVersion [2,4]) "." "data/**/*.hs" >>= traverse_ putStrLn

[cdepillabout]

@quasicomputational Here's the output of cabal new-run test.hs in the ormolu directory:

https://gist.github.com/cdepillabout/9a2ca42fcf78fa9462b63ea3c4a8f533#file-output-of-test-hs-txt

It looks like it is listing all the files correctly, so I'm not sure what could be going on here :-\

[cdepillabout]

@quasicomputational

Here's the function you linked to:

cabal/Cabal/Distribution/Simple/Install.hs

Lines 224 to 236 in ed3ae13

	installDataFiles :: Verbosity -> PackageDescription -> FilePath -> IO ()
	installDataFiles verbosity pkg_descr destDataDir =
	flip traverse_ (dataFiles pkg_descr) $ \ file -> do
	let srcDataDirRaw = dataDir pkg_descr
	srcDataDir = if null srcDataDirRaw
	then "."
	else srcDataDirRaw
	files <- matchDirFileGlob verbosity (specVersion pkg_descr) srcDataDir file
	let dir = takeDirectory file
	createDirectoryIfMissingVerbose verbosity True (destDataDir </> dir)
	sequence_ [ installOrdinaryFile verbosity (srcDataDir </> file')
	(destDataDir </> file')
	| file' <- files ]

Particularly these lines:

cabal/Cabal/Distribution/Simple/Install.hs

Lines 231 to 233 in ed3ae13

	files <- matchDirFileGlob verbosity (specVersion pkg_descr) srcDataDir file
	let dir = takeDirectory file
	createDirectoryIfMissingVerbose verbosity True (destDataDir </> dir)

And here is the relevant line from your test.hs file:

main =
  matchDirFileGlob normal (mkVersion [2,4]) "." "data/**/*.hs" >>= traverse_ putStrLn

In Cabal.Distribution.Simple.Install, if file is just the filepath with globs (in this case data/**/*.hs, then maybe takeDirectory file will return something like data/**? Then when createDirectoryIfMissing is called, it tries to literally create data/**?

[quasicomputational]

So, that makes sense why this is failing. And I just worked out why I couldn't replicate it: I had the wrong branch cheked out. Whoops. Thanks for the remote debugging support!

I'll put together a test and a patch for this.

(Also, this bug would've been caught if globs were a newtype instead of FilePath. I had that idea on the backburner for a while, but seeing it actually cause problems definitely gives it a bump.)

[mrkkrp]

@quasicomputational So when we'll be able to use the fix?

[quasicomputational]

Hopefully soon, when Cabal 3.0 comes out with GHC 8.8! But one snag is that users will still have 2.4 installs for a while, so, yeah.

[23Skidoo]

I can make one final lib:Cabal 2.4 point release if this helps.

[mrkkrp]

If that will allow users to build cabal install without the bug, that would be awesome.

[mrkkrp]

@23Skidoo Is that release coming?