My last Mac OS update replaced Python, which as usual broke the virtualenvs I use and when building a new virtualenv it seemed like as good a time as any to upgrade to Python 3 and Django 2.0. This lead to a lot of ‘adventures’ as I discovered how well Python 3 is or isn’t supported in my setup.
Part 1 of the Running to Stand Still series.
This mostly involved running
2to3 then changing back all the mistaken
changes, such as unwrapping the occasional needless
list(...) applied to iterators that were used in iteration,
and renaming any attribute named
next wrongly renamed to
I experimented with using Pipenv for this but switched back to Pip-Tools
because my server looks for
requirements.txt file and I don’t want to change
that just yet.
Just to be thorough I created a PyPy3 virtualenv as well. This flushed out
two things. First, it implements Python 3.5’s library rather than 3.6, so a
couple of places needed
encode calls added where 3.6 does
Second, I ended up replacing
documentation claims that the latter will use fast modules if available, so I
assume it incorporates the equivalent of
lxml, and this frees me from trying
lxml to build for PyPy.
On the server
On the server it was easy enough to download the latest Python source and do
make to build it and install in
(Hopefully system utilities using Python will carry on working with the
I was able to update Pip to version 10 (from something ridiculous like version 1.4) and hence upgrade to new version of virtualenv.
Switching to the
alleged user (the account that owns the alleged website’s
files on the server) I created a new Python-3 env.
I had to reinstall Fabric because it is stuck on Python 2.
Fabric is a command invoker tool I use to deploy my site to the server: it has conventions for running shell commands either remotely or locally and dealing with the results. I had been using it from a copy installed in teh same virtualenv as the rest of the blog software.
Unfortunately the 1.x version of Fabric cannot run on a Python 3 system: it hasn’t been ported. Instead Fabric 2, a ground-up rewrite will support Python 3. But alas! the new Fabric, while having a cleaner architecture and so on, does not run Fabric 1 scripts, and porting them is not trivial and possibly not yet possible, depending on whether the features you use have been reimplemented yet.
The fix (for now) has been to install Fabric globally, instead of using one in the virtualenv I use for the site.
The next show-stopper was a surprise: uWSGI, despite having virtualenv-savvy features, does not actually use the virtualenv. As a result, uWSGI compiled for Python 2 cannot run servers that run in Python 3: the log fills up with nonsensical import errors.
The solution is a story for another time.