Running Haskell script with Stack uses multiple versions of one package

[pharpend]

Hello, there

In Snowdrift, we have a database setup script called sdb.hs. It uses
stack along with the turtle package to run as a quasi-shell script.

When I tried to run it today to start the database server, I got this
error message:

sdb.hs:70:47:
    Couldn't match type ‘T.Text’
                   with ‘Text’
    NB: ‘T.Text’
          is defined in ‘Data.Text.Internal’ in package ‘text-1.2.2.1’
        ‘Text’ is defined in ‘Data.Text.Internal’ in package ‘text-1.2.2.0’
    Expected type: String -> Text
      Actual type: String -> T.Text
    In the first argument of ‘map’, namely ‘T.pack’
    In the second argument of ‘procs’, namely ‘(map T.pack as')’

sdb.hs:140:37:
    Couldn't match type ‘T.Text’
                   with ‘Text’
    NB: ‘T.Text’
          is defined in ‘Data.Text.Internal’ in package ‘text-1.2.2.1’
        ‘Text’ is defined in ‘Data.Text.Internal’ in package ‘text-1.2.2.0’
    Expected type: Char -> Text
      Actual type: Char -> T.Text
    In the first argument of ‘(<$>)’, namely ‘T.singleton’
    In the second argument of ‘(>>)’, namely
      ‘(T.singleton <$> (char '#' <|> newline))’

sdb.hs:221:41:
    Couldn't match type ‘T.Text’
                   with ‘Text’
    NB: ‘T.Text’
          is defined in ‘Data.Text.Internal’ in package ‘text-1.2.2.1’
        ‘Text’ is defined in ‘Data.Text.Internal’ in package ‘text-1.2.2.0’
    Expected type: Char -> Text
      Actual type: Char -> T.Text
    In the second argument of ‘(.)’, namely ‘T.singleton’
    In the first argument of ‘fmap’, namely ‘(Just . T.singleton)’

As you can see, stack is using multiple versions of text, and coming up with this ridiculous error.

Bulleted information:

• Script URL: https://git.gnu.io/pharpend/snowdrift/blob/mech-sibling/sdb.hs
• stack --version:

Version 1.0.5, Git revision 51492566184b4f98106de0e8199c8962dace567e x86_64

• Steps to reproduce:

git clone https://git.gnu.io/pharpend/snowdrift.git -b mech-sibling
cd snowdrift
./sdb.hs init

[silky]

interestingly following your steps i can't reproduce this error, and i have the same version of stack

03:02 PM noon ∈ dev>stack --version
Version 1.0.5, Git revision 51492566184b4f98106de0e8199c8962dace567e x86_64

is there something else going on? can you provide more information about your system?

[pharpend]

It's reasonably up-to-date (updated yesterday) Arch Linux. uname -a:

 Linux valentine 4.4.5-1-custom #1 SMP PREEMPT Sun Mar 27 13:50:03 MDT 2016 x86_64 GNU/Linux

PostgreSQL version 9.5.1-2.

~/.stack/global/stack.yaml:

flags: {}
packages: []
extra-deps: []
resolver: lts-3.7

[pharpend]

I tried it on a fresh Ubuntu VM, couldn't reproduce it either. I was able to reproduce it in a separate directory on Arch. I'm going to work some more to do so. Maybe add a stack setup && stack build step somewhere? I'm currently sitting through the world's longest stack build command.

[pharpend]

Updating the resolver in stack.yaml seemed to fix it on my Arch machine, somehow.

I don't want to close this until we figure out what went wrong, though. there is clearly a bug somewhere.

I opened a merge request at Snowdrift to update the resolver: https://git.gnu.io/snowdrift/snowdrift/merge_requests/177.

[mgsloan]

I don't think there's necessarily a bug here. The root cause is that runghc doesn't hide all the packages and tell ghc to explicitly use specific versions (see #1208). So, usually the existence of multiple versions of the same package is invisible to the user.

AFAIK, stack only has duplicate versions of the package when it exists in the global DB. Have you used cabal with your package databases? The shadowing of package DBs should cause a global DB version of text to fail, though. What does stack exec -- describe text say? It should give multiple descriptions of the package. I'm particularly interested in the package installation location.

Have you used cabal-install with your package DBs? That's one way this could happen.

[chreekat]

As the author of the sdb.hs in question, let me know if there's something I could do differently.

Can I force stack to use the package-local sandbox when run as a shebang interpreter? This script is only designed to be used with the Snowdrift project anyway. (Though I suppose it might be useful to other Yesoders... it makes a ton of sense to me that people would use a local cluster for dev, rather than depending on a system-level instance of Postgres.)

• Edit: of course, it's not useful to anyone currently since it has a bunch of Snowdrift-specific stuff baked in. :)

[mitchellwrosen]

I just ran into this myself.

[1 of 1] Compiling Main             ( insert.hs, insert.o )

insert.hs:11:40:
    Couldn't match type ‘System.Random.MWC.Gen
                           (Control.Monad.Primitive.PrimState IO)’
                   with ‘mwc-random-0.13.3.2:System.Random.MWC.Gen
                           (Control.Monad.Primitive.PrimState IO)’
    NB: ‘System.Random.MWC.Gen’
          is defined in ‘System.Random.MWC’ in package ‘mwc-random-0.13.4.0’
        ‘mwc-random-0.13.3.2:System.Random.MWC.Gen’
          is defined in ‘System.Random.MWC’ in package ‘mwc-random-0.13.3.2’
    Expected type: mwc-random-0.13.3.2:System.Random.MWC.GenIO
      Actual type: System.Random.MWC.GenIO

Verbose output of stack -v runghc insert.hs

Version 1.0.4 x86_64
2016-04-04 18:09:53.131443: [debug] Checking for project config at: /home/mitchell/sentenai/stream-simulation/stack.yaml @(stack_COhPD0SUWBs7s1VUFLBNcf:Stack.Config src/Stack/Config.hs:761:9)
2016-04-04 18:09:53.131906: [debug] Loading project config file stack.yaml @(stack_COhPD0SUWBs7s1VUFLBNcf:Stack.Config src/Stack/Config.hs:779:13)
2016-04-04 18:09:53.135487: [debug] Run process: ldd /home/mitchell/.local/bin/stack @(stack_COhPD0SUWBs7s1VUFLBNcf:System.Process.Read src/System/Process/Read.hs:269:3)
2016-04-04 18:09:53.151454: [debug] Trying to decode /home/mitchell/.stack/build-plan-cache/x86_64-linux/lts-5.1.cache @(stack_COhPD0SUWBs7s1VUFLBNcf:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:55:5)
2016-04-04 18:09:53.172857: [debug] Success decoding /home/mitchell/.stack/build-plan-cache/x86_64-linux/lts-5.1.cache @(stack_COhPD0SUWBs7s1VUFLBNcf:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:64:13)
2016-04-04 18:09:53.173615: [debug] Trying to decode /home/mitchell/.stack/indices/Hackage/00-index.cache @(stack_COhPD0SUWBs7s1VUFLBNcf:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:55:5)
2016-04-04 18:09:53.443407: [debug] Success decoding /home/mitchell/.stack/indices/Hackage/00-index.cache @(stack_COhPD0SUWBs7s1VUFLBNcf:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:64:13)
2016-04-04 18:09:53.467318: [debug] Run process: ghc --info @(stack_COhPD0SUWBs7s1VUFLBNcf:System.Process.Read src/System/Process/Read.hs:269:3)
2016-04-04 18:09:53.534058: [debug] Run process: ghc --numeric-version @(stack_COhPD0SUWBs7s1VUFLBNcf:System.Process.Read src/System/Process/Read.hs:269:3)
2016-04-04 18:09:53.575019: [debug] Run process: ghc-pkg --no-user-package-db field --simple-output Cabal version @(stack_COhPD0SUWBs7s1VUFLBNcf:System.Process.Read src/System/Process/Read.hs:269:3)
2016-04-04 18:09:53.631077: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack_COhPD0SUWBs7s1VUFLBNcf:System.Process.Read src/System/Process/Read.hs:269:3)
2016-04-04 18:09:53.687518: [debug] Run process: runghc insert.hs @(stack_COhPD0SUWBs7s1VUFLBNcf:Stack.Exec src/Stack/Exec.hs:51:5)

Weirdly, stack runghc looks at my stack.yaml and proceeds to do nothing with it? That is,

$ stack list-dependencies | grep mwc-random
mwc-random 0.13.3.2

yet

$ ghc-pkg --no-user-package-db list --global | grep mwc-random
    mwc-random-0.13.4.0

Now, I have no idea why I have mwc-random-0.13.4.0 installed in the global package database, but isn't the resolver lts-5.1 supposed to override that?

Thanks.

[mgsloan]

It really should. On the surface, it looks like ghc is preferring the global DB, for some bizarre reason.

I asked some diagnostic questions in an early comment, and got no reply. I will reiterate them. Anyone affected by this, feel free to provide the output:

The shadowing of package DBs should cause a global DB version of text to fail, though. What does stack exec -- describe text say? It should give multiple descriptions of the package. I'm particularly interested in the package installation location.

Have you used cabal-install with your package DBs?

Furthermore, are you using a system install of ghc? I have a suspicion that this is much less likely to happen with stack managed ghc installs, because you're less likely to use cabal-install on the global DB

[mitchellwrosen]

Ah, I didn't catch that comment. What is describe? I'm on arch linux and there is no describe package.

[mgsloan]

Oh I meant stack exec -- ghc-pkg describe text

[mitchellwrosen]

Ah, here you go:

name: mwc-random
version: 0.13.3.2
import-dirs: /home/mitchell/.stack/snapshots/x86_64-linux/lts-5.1/7.10.3/lib/x86_64-linux-ghc-7.10.3/mwc-random-0.13.3.2-0SEz8XRK7wOF4DF0uaF48y
library-dirs: /home/mitchell/.stack/snapshots/x86_64-linux/lts-5.1/7.10.3/lib/x86_64-linux-ghc-7.10.3/mwc-random-0.13.3.2-0SEz8XRK7wOF4DF0uaF48y
data-dir: /home/mitchell/.stack/snapshots/x86_64-linux/lts-5.1/7.10.3/share/x86_64-linux-ghc-7.10.3/mwc-random-0.13.3.2
hs-libraries: HSmwc-random-0.13.3.2-0SEz8XRK7wOF4DF0uaF48y
depends:
    base-4.8.2.0-0d6d1084fbc041e1cded9228e80e264d
    primitive-0.6.1.0-b2a7b9f8d5591c0d4ce7ef238a3217d2
    time-1.5.0.1-edbd1a50e7922b396ada189ab8e8523b
    vector-0.11.0.0-299aefb173ce5a731565d31f609a0cfd
haddock-interfaces: /home/mitchell/.stack/snapshots/x86_64-linux/lts-5.1/7.10.3/doc/mwc-random-0.13.3.2/mwc-random.haddock
haddock-html: /home/mitchell/.stack/snapshots/x86_64-linux/lts-5.1/7.10.3/doc/mwc-random-0.13.3.2
pkgroot: "/home/mitchell/.stack/snapshots/x86_64-linux/lts-5.1/7.10.3"
---
name: mwc-random
version: 0.13.4.0
import-dirs: /usr/lib/ghc-7.10.3/site-local/mwc-random-0.13.4.0
library-dirs: /usr/lib/ghc-7.10.3/site-local/mwc-random-0.13.4.0
data-dir: /usr/share/x86_64-linux-ghc-7.10.3/mwc-random-0.13.4.0
hs-libraries: HSmwc-random-0.13.4.0-0lrQ1SkkNA85sa8eZ98xQk
depends:
    base-4.8.2.0-0d6d1084fbc041e1cded9228e80e264d
    primitive-0.6.1.0-b2a7b9f8d5591c0d4ce7ef238a3217d2
    time-1.5.0.1-edbd1a50e7922b396ada189ab8e8523b
    vector-0.11.0.0-299aefb173ce5a731565d31f609a0cfd
haddock-interfaces: /usr/share/doc/haskell-mwc-random/html/mwc-random.haddock
haddock-html: /usr/share/doc/haskell-mwc-random/html
pkgroot: "/usr/lib/ghc-7.10.3"

I do have a system ghc install, but only because the Arch Linux stack package depends on it. I've never used cabal-install to install anything globally.

[pharpend]

Here's what happens when I run stack exec -- ghc-pkg describe text:

name: text
version: 1.2.2.1
id: text-1.2.2.1-d0adb978563e9f52dc308d1d0db7212c
key: text_HmqVQnZSpjaC156ABqPhne
license: BSD3
copyright: 2009-2011 Bryan O'Sullivan, 2008-2009 Tom Harper
maintainer: Bryan O'Sullivan <bos@serpentine.com>
homepage: https://github.com/bos/text
synopsis: An efficient packed Unicode text type.
description:
    .
    An efficient packed, immutable Unicode text type (both strict and
    lazy), with a powerful loop fusion optimization framework.
    .
    The 'Text' type represents Unicode character strings, in a time and
    space-efficient manner. This package provides text processing
    capabilities that are optimized for performance critical use, both
    in terms of large data quantities and high speed.
    .
    The 'Text' type provides character-encoding, type-safe case
    conversion via whole-string case conversion functions. It also
    provides a range of functions for converting 'Text' values to and from
    'ByteStrings', using several standard encodings.
    .
    Efficient locale-sensitive support for text IO is also supported.
    .
    These modules are intended to be imported qualified, to avoid name
    clashes with Prelude functions, e.g.
    .
    > import qualified Data.Text as T
    .
    To use an extended and very rich family of functions for working
    with Unicode text (including normalization, regular expressions,
    non-standard encodings, text breaking, and locales), see
    the @text-icu@ package:
    <http://hackage.haskell.org/package/text-icu>
category: Data, Text
author: Bryan O'Sullivan <bos@serpentine.com>
exposed: True
exposed-modules:
    Data.Text Data.Text.Array Data.Text.Encoding
    Data.Text.Encoding.Error Data.Text.Foreign Data.Text.IO
    Data.Text.Internal Data.Text.Internal.Builder
    Data.Text.Internal.Builder.Functions
    Data.Text.Internal.Builder.Int.Digits
    Data.Text.Internal.Builder.RealFloat.Functions
    Data.Text.Internal.Encoding.Fusion
    Data.Text.Internal.Encoding.Fusion.Common
    Data.Text.Internal.Encoding.Utf16 Data.Text.Internal.Encoding.Utf32
    Data.Text.Internal.Encoding.Utf8 Data.Text.Internal.Functions
    Data.Text.Internal.Fusion Data.Text.Internal.Fusion.CaseMapping
    Data.Text.Internal.Fusion.Common Data.Text.Internal.Fusion.Size
    Data.Text.Internal.Fusion.Types Data.Text.Internal.IO
    Data.Text.Internal.Lazy Data.Text.Internal.Lazy.Encoding.Fusion
    Data.Text.Internal.Lazy.Fusion Data.Text.Internal.Lazy.Search
    Data.Text.Internal.Private Data.Text.Internal.Read
    Data.Text.Internal.Search Data.Text.Internal.Unsafe
    Data.Text.Internal.Unsafe.Char Data.Text.Internal.Unsafe.Shift
    Data.Text.Lazy Data.Text.Lazy.Builder Data.Text.Lazy.Builder.Int
    Data.Text.Lazy.Builder.RealFloat Data.Text.Lazy.Encoding
    Data.Text.Lazy.IO Data.Text.Lazy.Internal Data.Text.Lazy.Read
    Data.Text.Read Data.Text.Unsafe
hidden-modules: Data.Text.Show
trusted: False
import-dirs: /usr/lib/ghc-7.10.3/site-local/text-1.2.2.1
library-dirs: /usr/lib/ghc-7.10.3/site-local/text-1.2.2.1
data-dir: /usr/share/x86_64-linux-ghc-7.10.3/text-1.2.2.1
hs-libraries: HStext-1.2.2.1-HmqVQnZSpjaC156ABqPhne
depends:
    array-0.5.1.0-960bf9ae8875cc30355e086f8853a049
    base-4.8.2.0-0d6d1084fbc041e1cded9228e80e264d
    binary-0.7.5.0-5784fd031a720c3b84e73006e444c7ca
    bytestring-0.10.6.0-c60f4c543b22c7f7293a06ae48820437
    deepseq-1.4.1.1-614b63b36dd6e29d2b35afff57c25311
    ghc-prim-0.4.0.0-6cdc86811872333585fa98756aa7c51e
    integer-gmp-1.0.0.0-3c8c40657a9870f5c33be17496806d8d
haddock-interfaces: /usr/share/doc/haskell-text/html/text.haddock
haddock-html: /usr/share/doc/haskell-text/html
pkgroot: "/usr/lib/ghc-7.10.3"

I'm also on Arch Linux, FWIW. I have used this installation since before stack existed, so there are several packages installed with cabal-install.

For me, updating the resolver fixed this for some reason. No idea why.

[mitchellwrosen]

@pharpend Doesn't seem to be a problem there; ghc-pkg only sees one global installation of text. You may want to trim down that output (or delete it) for that reason - it's noise :)

[luigy]

For me, updating the resolver fixed this for some reason. No idea why.

Hmm I believe your compiler changed from 7.10.2 to 7.10.3 when updating the resolver from lts-3.7 to lts-5ish and therefore different global pkgdb

testing instead stack --resolver lts-3.7 exec -- ghc-pkg describe text would probably reveal more about what was going on before

It really should. On the surface, it looks like ghc is preferring the global DB, for some bizarre reason.

indeed! curious to see if describe reveals anything different this time