support optional dependencies spec from conda 4.4

[msarahan]

fixes #1964

[msarahan]

@kalefranz please review - especially make sure that the test case adequately captures the functionality that you had in mind.

[kalefranz]

LGTM

[kalefranz]

codecov has been giving me problems recently on Appveyor too. I pinned to pip install codecov==2.0.5 a few days ago. That seemed to help.

[msarahan]

Thanks. Merging.

[183amir]

There is this option in conda build:

build:
  pin_depends: strict

Can this be used for run_constrained too? Or is something similar available?

[msarahan]

no, pin_depends is something completely different. See this for its origin: #741

The idea for it was to constrain dependencies tightly. optional dependencies are completely different - they say "if this dependency is installed, make sure it meets these additional constraints"

[183amir]

So I have this recipe:

package:
  name: bob-devel
  version: {{ os.environ.get('CI_BUILD_TAG', 'vx.y.z')[1:] }}

build:
  number: 0
  pin_depends: strict

requirements:
  build:
    - numpy x.x
  run:
    - python
    - numpy x.x
    # Our dependency list.
    # Everything is pinned to allow better reproducibility.
    # The pinnings below are coming from anaconda 4.4.0
    - curl 7.52.1
    - cython 0.25.2
    - cytoolz 0.8.2
    - dask 0.14.3
    - freetype 2.5.5
    - hdf5 1.8.17
    - imagesize 0.7.1
    - ipython 5.3.0
    - jpeg 9b
    - libpng 1.6.27
    - libtiff 4.0.6
    - matplotlib 2.0.2
    - mkl 2017.0.1
    - nose 1.3.7
    - numba 0.33.0
    - numba 0.33.0
    - numexpr 2.6.2
    - pandas 0.20.1
    - pillow 4.1.1
    - pip 9.0.1
    - scikit-image 0.13.0
    - scikit-learn 0.18.1
    - scipy 0.19.0
    - setuptools 27.2.0
    - six 1.10.0
    - sphinx 1.5.6
    - sqlalchemy 1.1.9
    - toolz 0.8.2
    # these pinnings of indirect dependencies make sure bob-devel is compatible with anaconda 4.4.0, hopefully
    - bokeh 0.12.5
    - nbconvert 5.1.1
    - testpath 0.3
    # the rest of the dependencies
    - boost 1.61.0
    - caffe 1.0.0rc5  # [linux]
    - cmake 3.6.3
    - coverage 4.3.4
    - cyvlfeat 0.4.5
    - dlib 19.0
    - docopt 0.6.2
    - ffmpeg 2.8.10
    - gcc 4.*  # [linux]
    - giflib 5.1.4
    - ipdb 0.10.2
    - kaldi r7271.1a4dbf6  # [linux]
    - keras 2.0.8  # [linux]
    - libblitz 0.10
    - libgcc 4.*  # [linux]
    - libmatio 1.5.6
    - libogg 1.3.2
    - libsvm 3.21
    - madmom 0.15.1
    - menpo 0.8.1
    - menpofit 0.5.0
    - menpowidgets 0.3.0p6
    - mne 0.14.1
    - opencv 3.1.0
    - pkg-config 0.28
    - pkgtools 0.7.3
    - pyedflib 0.1.9
    - schema 0.6.5
    - sox 14.4.2
    - sphinx_rtd_theme 0.2.4
    - tensorflow 1.2.1  # [linux]
    - toolchain 2.3.2
    - virtualenv 15.1.0
    - vlfeat 0.9.20
    - zc.buildout 2.9.2

test:
  imports:
    - tempfile

about:
  home: http://www.idiap.ch/software/bob
  license: BSD 3-Clause
  summary: A meta-package to define an environment for Bob package development

extra:
  recipe-maintainers:
    - anjos
    - amohammadi

And I pin the versions that I want in it. But the great thing about this is that it pins the indirect dependencies too. It's like taking a snapshot of the environment while only pinning the relevant ones in the build dependencies.

Now what I really want is actually this frozen environment (with direct and indirect pins) to be in run_constrained instead of run. This way I don't have to install all the dependencies when I install bob-devel but if in future I decide to install one of those dependencies in my environment, I will get a version that is mandated by bob-devel.

Does that make sense?

[msarahan]

Yes, I can see the draw in that. I'd gladly consider a PR. It may also be better to do this with configuration on the conda side somehow - just depends which way is easier to share with other people.

[183amir]

Is it possible to do this with conda configurations?
I would also see that making a package like bob-devel a build dependency as well would be useful.

[msarahan]

I think it's possible, but you should file an issue on the conda tracker and ask for examples and docs.

[kalefranz]

If I understand correctly, this is exactly how the pinned_packages configuration parameter works in conda 4.4 now. Note you can have an environment-only .condarc file (at the prefix root of the environment). The --env flag on conda config also manipulates this file.

[183amir]

Hi @kalefranz while having this as the conda configuration is useful. I don't think you can benefit from that during the conda-build process. Right now, with the introduction of the variants, it is very easy to pin direct dependencies during build time. But, it is not possible to pin the indirect recipes during build time. But I am thinking that I can achieve this with the new run_constrained feature.

[isuruf]

Is this documented somewhere?

[jakirkham]

Also not seeing docs. Is there an issue for this or am I just missing it?

[github-actions]

Hi there, thank you for your contribution!

This pull request has been automatically locked because it has not had recent activity after being closed.

Please open a new issue or pull request if needed.

Thanks!