The shitty python dependency management

For java, we have maven, gradle. for nodejs, we have npm, npm is so good, that why I feel python package management is so ugly.

pip install dependency globally by default

Although python3 have venv, you need activate the venv and then the pip will install package in that venv.

Aslo you can use –target to specify a folder.

All these seems to like many patch on a old clothes – ugly.

pipenv

Pipenv is a tool that aims to bring the best of all packaging worlds (bundler, composer, npm, cargo, yarn, etc.) to the Python world. Windows is a first–class citizen, in our world.

It automatically creates and manages a virtualenv for your projects, as well as adds/removes packages from your Pipfile as you install/uninstall packages. It also generates the ever–important Pipfile.lock, which is used to produce deterministic builds.

This is the one I need, the project have over 12K stars on github.

Basic usage

Install pipenv

sudo pip install pipenv (windows need sudo, need scoop to install sudo command)

for Ubuntu 16 python3:
sudo apt install python3
sudo apt install python3-pip
pip3 install --user pipenv

More about installation on windows 10

I have tried many ways to resolve problems when install pipenv on windows when I have to code on sh()t windows and sh()t python:

Install choco

Install python via choco

Why?! Because it will save your life by choco, it will automatically set path for your powershell to avoid can not found python,pip …, and also it will benifit your VSCode.

open powershell in administrator mode, run choco install python --version=3.6.7

Install pipenv

Open a normal powershell run, pip uninstall pipenv, then run pip install pipenv. anyway, you’d better uninstall at first then install.

Resolve decode problem

When you encounter problem like File "c:\python37\lib\site-packages\pipenv\vendor\shellingham\nt.py", line 78, in _iter_process info = {'executable': str(pe.szExeFile.decode('utf-8'))} when run pipenv Open the nt.py file, change the code to info = {'executable': str(pe.szExeFile.decode('utf-8', errors='ignore'))}

Setup a pipenv project

Go to the root folder of the python project. run pipenv --three for a python3 project run pipenv --two for a python2 project run pipenv --python 3.6 for a python3.6 version project

Like npm install

pipenv install

Like yarn add

pipenv install <packagename> pipenv install <packagename>=<versionname> --dev

Load the shell

pipenv shell

Like yarn remove

pipenv uninstall <packagename> pipenv uninstall all

See the graph

pipenv graph

generate a requirements.txt

pipenv lock -r > requirements.txt

Remove pipenv for vcurrent project

pipenv –rm

VSCode setting

To let VSCode understand the venv created by pipenv, see Configuring Pipenv in Visual Studio Code
pipenv --venv to show the venv path
then in settings.json under the .vscode directory of the project windows:

{
    "python.pythonPath": "<venvpath>/Scripts/python.exe"
}

To let VSCode load pylint for Django, see Class has no objects member
pipenv install pylint-django --dev
then in settings.json under the .vscode directory of the project windows:

{"python.linting.pylintArgs": [
     "--load-plugins=pylint_django"
],}

The mysqlclient problem

pipenv is like a life saver – But! it still can not resolve such a problem: Can’t open ‘mysql.h’ - and Can’t open : ‘config-win.h’: No such file or directory
When I use command pipenv install mysqlclient, this problem happen.

I check mysql version by running sql SELECT version(); , it shows 5.7, not a problem, then I read the mysqlclient document, hmm, it need install mysqlconnector/c, but I remember I have installed it, I run the installer again and confirmed it, what’s going on?

Then I download the latest installer, but I can not found the option of connector/c in the installer wizard, what? Are you kidding me - Oracle?

Use wheel

As the mysqlclient suggested, we can use a wheel file to install directly. I download thew wheel file from https://www.lfd.uci.edu/~gohlke/pythonlibs/#mysqlclient I try it by running pipenv install <pathotthedownloadedwheelfilepath>, it throw error mysqlclient-1.3.13-cp37-cp37m-win32.whl is not a supported wheel on this platform. tried amd_64, not working too …

I found I am going to back to the old way of compile a c project.

Found the same problem at Python Connector for Django 1.9 and Python 3.5? [closed], the python version I running is 3.6(64 bit).

The root cause is > mysqlclient can not currently be compiled on Python 3.5 for Windows because the MySQL Connector/C is not yet compatible with the Visual Studio 2015 compiler required by Python 3.5. The solution is use mysqlclient 1.3 which have add support for python3.3~

Then I tried pipenv install mysqlclient==1.3.12 it works!!!!!

The final solution

• make sure mysql connector/c is installed(the old version like 5.7 of windows mysql server installer have this option)
• pipenv install mysqlclient==1.3.12

The pipenv install slow problem

Mostly, it is because the pipenv did not show any progress of the downloading. As the pipenv using pip cache, and pip have the progress showing. we can download the dependency at first by pip and then run pipenv install – hope pipenv resolve this prolem ASAP. python3 -m install tensorflow pipenv install tensorflow this still not working, so if you encounter this problem, run pipenv install --skip-lock will download the dependency but will not create the lock file.

Refs

Why is python dependency management so shit?

Better Python dependency while packaging your project