Homebrew on Mac and Python’s mysqlclient

I’ve spent the last couple of hours struggling with trying to get the python package mysqlclient to install on my development machine — a mac book pro.

On my previous mac it was as simple as

pip install mysqlclient

Unfortunately on this box, that caused an error

_mysql.c:29:10: fatal error: 'my_config.h' file not found
    #include "my_config.h"
             ^
    1 error generated.
    error: command 'clang' failed with exit status 1

it seems that the mariadb that I had installed with homebrew did not install the headers required to compile the binary part of mysqlclient

Looking at the github repo for mysqlclient, it seems they recommend mysql-connector-c be installed. Well homebrew will not install that with mariadb installed so I had to do this

brew unlink mariadb

Then I tried again after running

brew install mysql-connector-c

but it turns out that mysql-connector-c as installed by homebrew is borked and I then got this:

Collecting mysqlclient
  Using cached mysqlclient-1.3.10.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/5n/zxxhq1_91dj_4jzxy6zbz7yh000ggl/T/pip-build-g59txd77/mysqlclient/setup.py", line 17, in <module>
        metadata, options = get_config()
      File "/private/var/folders/5n/zxxhq1_91dj_4jzxy6zbz7yh000ggl/T/pip-build-g59txd77/mysqlclient/setup_posix.py", line 54, in get_config
        libraries = [dequote(i[2:]) for i in libs if i.startswith('-l')]
      File "/private/var/folders/5n/zxxhq1_91dj_4jzxy6zbz7yh000ggl/T/pip-build-g59txd77/mysqlclient/setup_posix.py", line 54, in <listcomp>
        libraries = [dequote(i[2:]) for i in libs if i.startswith('-l')]
      File "/private/var/folders/5n/zxxhq1_91dj_4jzxy6zbz7yh000ggl/T/pip-build-g59txd77/mysqlclient/setup_posix.py", line 12, in dequote
        if s[0] in "\"'" and s[0] == s[-1]:
    IndexError: string index out of range
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/5n/zxxhq1_91dj_4jzxy6zbz7yh000ggl/T/pip-build-g59txd77/mysqlclient/

Well it turns out that the reason for this is that mysql_config that is included with this generates bad output, adding a -l at the end that mysqlclient expects to be followed by a parameter

So I figured, why not see if I can install the mariadb version

brew uninstall mysql-connector-c
brew install mariadb-connector-c
pip install mysqlclient

No luck:

Collecting mysqlclient
  Using cached mysqlclient-1.3.10.tar.gz
    Complete output from command python setup.py egg_info:
    /bin/sh: mysql_config: command not found
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/5n/zxxhq1_91dj_4jzxy6zbz7yh000ggl/T/pip-build-k3y0nckz/mysqlclient/setup.py", line 17, in <module>
        metadata, options = get_config()
      File "/private/var/folders/5n/zxxhq1_91dj_4jzxy6zbz7yh000ggl/T/pip-build-k3y0nckz/mysqlclient/setup_posix.py", line 44, in get_config
        libs = mysql_config("libs_r")
      File "/private/var/folders/5n/zxxhq1_91dj_4jzxy6zbz7yh000ggl/T/pip-build-k3y0nckz/mysqlclient/setup_posix.py", line 26, in mysql_config
        raise EnvironmentError("%s not found" % (mysql_config.path,))
    OSError: mysql_config not found

not entirely surprisingly mysql_config is not installed, however mariadb_config is.

ls -l /usr/local/bin/mariadb_config
lrwxr-xr-x  1 james-har  admin  54 29 Aug 20:18 /usr/local/bin/mariadb_config -> ../Cellar/mariadb-connector-c/2.2.2/bin/mariadb_config

I figured that these are probably compatible and since it doesn’t output the missing parameter it could probably be used as a drop in replacement

so using it should be as simple as adding a symlink

cd /usr/local/bin/
ln -s ../Cellar/mariadb-connector-c/2.2.2/bin/mariadb_config mysql_config

Nope yet more problems

Collecting mysqlclient
  Using cached mysqlclient-1.3.10.tar.gz
Building wheels for collected packages: mysqlclient
  Running setup.py bdist_wheel for mysqlclient ... error
  Complete output from command /Users/james-har/venv/cetaapi/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/private/var/folders/5n/zxxhq1_91dj_4jzxy6zbz7yh000ggl/T/pip-build-hmxlf6p2/mysqlclient/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /var/folders/5n/zxxhq1_91dj_4jzxy6zbz7yh000ggl/T/tmplaracsfapip-wheel- --python-tag cp36:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-10.12-x86_64-3.6
  copying _mysql_exceptions.py -> build/lib.macosx-10.12-x86_64-3.6
  creating build/lib.macosx-10.12-x86_64-3.6/MySQLdb
  copying MySQLdb/__init__.py -> build/lib.macosx-10.12-x86_64-3.6/MySQLdb
  copying MySQLdb/compat.py -> build/lib.macosx-10.12-x86_64-3.6/MySQLdb
  copying MySQLdb/connections.py -> build/lib.macosx-10.12-x86_64-3.6/MySQLdb
  copying MySQLdb/converters.py -> build/lib.macosx-10.12-x86_64-3.6/MySQLdb
  copying MySQLdb/cursors.py -> build/lib.macosx-10.12-x86_64-3.6/MySQLdb
  copying MySQLdb/release.py -> build/lib.macosx-10.12-x86_64-3.6/MySQLdb
  copying MySQLdb/times.py -> build/lib.macosx-10.12-x86_64-3.6/MySQLdb
  creating build/lib.macosx-10.12-x86_64-3.6/MySQLdb/constants
  copying MySQLdb/constants/__init__.py -> build/lib.macosx-10.12-x86_64-3.6/MySQLdb/constants
  copying MySQLdb/constants/CLIENT.py -> build/lib.macosx-10.12-x86_64-3.6/MySQLdb/constants
  copying MySQLdb/constants/CR.py -> build/lib.macosx-10.12-x86_64-3.6/MySQLdb/constants
  copying MySQLdb/constants/ER.py -> build/lib.macosx-10.12-x86_64-3.6/MySQLdb/constants
  copying MySQLdb/constants/FIELD_TYPE.py -> build/lib.macosx-10.12-x86_64-3.6/MySQLdb/constants
  copying MySQLdb/constants/FLAG.py -> build/lib.macosx-10.12-x86_64-3.6/MySQLdb/constants
  copying MySQLdb/constants/REFRESH.py -> build/lib.macosx-10.12-x86_64-3.6/MySQLdb/constants
  running build_ext
  building '_mysql' extension
  creating build/temp.macosx-10.12-x86_64-3.6
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -Dversion_info=(1,3,10,'final',0) -D__version__=1.3.10 -I/usr/local/Cellar/mariadb-connector-c/2.2.2/include/mariadb -I/usr/local/Cellar/mariadb-connector-c/2.2.2/include/mariadb/mysql -I/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c _mysql.c -o build/temp.macosx-10.12-x86_64-3.6/_mysql.o
  clang -bundle -undefined dynamic_lookup -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk build/temp.macosx-10.12-x86_64-3.6/_mysql.o -L/usr/local/Cellar/mariadb-connector-c/2.2.2/lib/mariadb -lmariadb -lssl -ldl -lm -lpthread -o build/lib.macosx-10.12-x86_64-3.6/_mysql.cpython-36m-darwin.so /usr/lib/libiconv.dylib
  ld: library not found for -lssl
  clang: error: linker command failed with exit code 1 (use -v to see invocation)
  error: command 'clang' failed with exit status 1

a bit of googling and I found that I need to explicitly pass the homebrew OpenSSL paths as environment variables

env LDFLAGS="-I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib" pip install mysqlclient

which eventually led to the message I had been trying to get to for a couple of hours

Successfully installed mysqlclient-1.3.10

phew

Looking at this, it looks like it may be simpler in the next update of mysqlclient, I sure hope so.

Update

env LDFLAGS="-I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib" pip install mysqlclient==1.3.12

now works without any of the homebrew jumparounds suggested above. Thanks to the developers of that library for getting it sorted