web-dev-qa-db-fra.com

pip install AttributeError: _DistInfoDistribution__dep_map

J'essaie de faire tourner un conteneur nvidia-docker (2.0) dans Ubuntu 16.04 exécutant Conda avec quelques bibliothèques python (tensorflow activé par GPU, opencv et gdal) et leurs diverses dépendances.

Explication générale du problème

J'ai quelques bibliothèques que je dois installer en utilisant pip dans cet environnement (par exemple tensorflow-gpu 1.10.0 et quelques autres bibliothèques personnalisées), mais chaque fois que j'essaie d'installer un paquet en utilisant pip soit dans mon Dockerfile soit après, je obtenez l'erreur suivante:

Exception:
Traceback (most recent call last):
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2869, in _dep_map
    return self.__dep_map
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2663, in __getattr__
    raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 93, in __init__
    req = REQUIREMENT.parseString(requirement_string)
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1632, in parseString
    raise exc
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1622, in parseString
    loc, tokens = self._parse( instring, 0 )
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3395, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1383, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3183, in parseImpl
    raise ParseException(instring, loc, self.errmsg, self)
pip._vendor.pyparsing.ParseException: Expected stringEnd (at char 33), (line:1, col:34)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2949, in __init__
    super(Requirement, self).__init__(requirement_string)
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 97, in __init__
    requirement_string[e.loc:e.loc + 8]))
pip._vendor.packaging.requirements.InvalidRequirement: Invalid requirement, parse error at "'; extra '"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_internal/basecommand.py", line 141, in main
    status = self.run(options, args)
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 330, in run
    self._warn_about_conflicts(to_install)
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 456, in _warn_about_conflicts
    package_set, _dep_info = check_install_conflicts(to_install)
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 98, in check_install_conflicts
    package_set = create_package_set_from_installed()
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 41, in create_package_set_from_installed
    package_set[name] = PackageDetails(dist.version, dist.requires())
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2607, in requires
    dm = self._dep_map
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2871, in _dep_map
    self.__dep_map = self._compute_dependencies()
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2881, in _compute_dependencies
    reqs.extend(parse_requirements(req))
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2942, in parse_requirements
    yield Requirement(line)
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2951, in __init__
    raise RequirementParseError(str(e))
pip._vendor.pkg_resources.RequirementParseError: Invalid requirement, parse error at "'; extra '"

Cette même erreur se produit si j'essaie d'installer d'autres packages en utilisant pip au lieu de conda install (D'autres que j'ai essayés sont setuptools, h5py et plusieurs autres dont je ne me souviens pas)

Dockerfile

Voici le Dockerfile utilisé pour générer cela, jusqu'à l'endroit où l'erreur est levée (lors de l'installation de tensorflow):

FROM nvidia/cuda:9.0-devel-ubuntu16.04
LABEL maintainer "[deleted]"

# get correct version of CUDNN for my system's CUDA
ENV CUDNN_VERSION 7.3.0.29
LABEL com.nvidia.cudnn.version="${CUDNN_VERSION}"

RUN apt-get update && apt-get install -y --no-install-recommends \
            libcudnn7=$CUDNN_VERSION-1+cuda9.0 \
            libcudnn7-dev=$CUDNN_VERSION-1+cuda9.0 && \
    apt-mark hold libcudnn7 && \
    rm -rf /var/lib/apt/lists/*

# install underlying requirements
RUN apt-get update \
  && apt-get install -y --no-install-recommends \
    bc \
    bzip2 \
    ca-certificates \
    curl \
    git \
    libgl1 \
    jq \
    nfs-common \
    parallel \
    python-pip \
    python-wheel \
    python-setuptools \
    unzip \
    wget \
    build-essential \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/*

# install anaconda
RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh -O ~/miniconda.sh && \
    /bin/bash ~/miniconda.sh -b -p /opt/conda && \
    rm ~/miniconda.sh && \
    /opt/conda/bin/conda clean -tipsy && \
    ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh && \
    echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc && \
    echo "conda activate base" >> ~/.bashrc

ENV TINI_VERSION v0.16.1
ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /usr/bin/tini
RUN chmod +x /usr/bin/tini

# add conda to $PATH and create a conda environment
ENV PATH /opt/conda/bin:$PATH
RUN conda update conda && \
    conda config --remove channels defaults && \
    conda config --add channels conda-forge && \
    conda create -n tf_keras python=3.6 \
                    && echo "source activate tf_keras" > ~/.bashrc
ENV PATH /opt/conda/envs/tf_keras/bin:$PATH

Shell ["/bin/bash", "-c"]

# install required libraries (and some dependencies)
RUN conda install -n tf_keras \
              osmnx=0.7.3 \
              affine \
              pyproj \
              pyhamcrest=1.9.0 \
              cython \
              fiona \
              h5py \
              ncurses \
              jupyter \
              jupyterlab \
              ipykernel \
              libgdal \
              matplotlib \
              numpy \
              opencv \
              pandas \
              pillow \
              pip \
              scipy \
              scikit-image \
              scikit-learn \
              shapely \
              gdal \
              rtree \
              tqdm \
              pandas \
              geopandas \
              rasterio

# get tensorflow
ARG TENSORFLOW_VERSION=1.10.0
ARG TENSORFLOW_DEVICE=gpu
ARG TENSORFLOW_APPEND=_gpu
RUN source activate tf_keras && \
    pip --no-cache-dir install https://storage.googleapis.com/tensorflow/linux/${TENSORFLOW_DEVICE}/tensorflow${TENSORFLOW_APPEND}-${TENSORFLOW_VERSION}-cp36-cp36m-linux_x86_64.whl

La dernière commande RUN renvoie l'erreur.

Ce que j'ai essayé:

  • Différentes versions de pip (10.0.1 et 18.0)
  • pip upgrade pip Ou conda upgrade pip (Essayer de mettre à jour pip de cette façon génère la même erreur)
  • mise à jour de setuptools (lance la même erreur si j'essaye de le faire avec pip)
  • J'ai vérifié que le bon pip est utilisé en ajoutant echo $(which pip) && avant mes commandes pip install - il retourne le pip de l'environnement virtuel.

Je voudrais simplement installer tout en utilisant conda et l'ignorer, mais il y a quelques bibliothèques internes à mon groupe qui ne sont pas disponibles pour conda.

pythondockertensorflowpipanaconda
11
nweir

Vous devez rétrograder votre version de testpath:

conda install 'testpath<0.4'

Voir le problème ici: https://github.com/conda-forge/testpath-feedstock/issues/7

17
ostrokach