large stack build uses a lot of space in temp directory

[joeyh]

General summary/comments (optional)

When stack build needs to build a lot of dependencies, it uses a lot of disk space in the system temp directory. In particular, build directories for all packages are left there after the libraries have been installed into .stack-work, and are only cleaned up after the build finishes.

Steps to reproduce

stack build in git-annex source tree on a system with eg a 1.3 gb tmpfs for /tmp. (The default on a modern debian system with 4 gb of ram.)

Expected

successful build

Actual

Build will eventually use over a GB of space in /tmp/stacknnnn and fail. The temp directory is then freed, and re-running the build picks up where it left off, clearly not needing any of the stuff that was being kept in the temp directory.

Workaround

Set TMP to point to a directory with more space than /tmp

Suggested fix

Delete package build directories of dependencies after the copy/register step.

Stack version

1.1.2

I'm fairly sure current stack also behaves this way too. Looking at https://github.com/commercialhaskell/stack/blob/4ee027d2d93f42e790ff170c139094cc8dbb2373/src/Stack/Build/Execute.hs , withExecuteEnv is used to run the build plan and uses a single withSystemTempDir

Method of installation

apt-get

[nponeccop]

Stack 1.3.2 on ArchLinux behaves the same way.

du -sBM /tmp/stack30907/ and ls -1 /tmp/stack30907/ | wc -l constantly grow. In my case tmpfs eats up all swap space so last few pakage builds are very very slow (seem to run GC too much but I didn't really check that) and finally I get an OOM error.

[mgsloan]

I've pushed a fix for this using the suggested fix of

Delete package build directories of dependencies after the copy/register step.

Good idea @joeyh !