In the past, we did explicitly install pyyaml through PIP to get a
version newer than the one provided in Ubuntu 18.04. Since then we
changed the calling code to not rely on newer pyyaml features and
are thus able to rely on an older version of pyyaml.
The distro-provided version of pyyaml is built with C bindings, which
are significantly faster than the pip-installed version, which uses a
pure Python-implementation of the parser/dumper.
There's a long and painful discussion on the setuptools issues
list (I count 9 issues raised the day after the release). This is all
to do with how Debian/Ubuntu installs stuff with pip.
There's an "official" workaround that involves setting things in your
environment, but that looks a bit fiddly with the Azure pipelines
stuff and this has been such a disaster that I'm pretty certain the
setuptools maintainers will release something more sensible soon.
See e.g.
https://github.com/pypa/setuptools/issues/2350#issuecomment-683512526
for a careful description of what's going on.
Put all build dependencies into a CI-specific script
(ci/install-build-deps.sh). No effort has been made to make this script
suitable for non-CI users.
