RevSys Python Builds
What’s wrong with stock Python images?
The main image variety is too big when the user is operating in a context that involves multiple private registries.
It’s base layer is an Ubuntu image (with a few exceptions, this provides an as-installed-by-Ubuntu environment) which is, essentially, what one gets with a base minimal install of Ubuntu 16.04.
The alpine images are delightfully wee. But, alas, musl. It is awesome. I mean, it exists. If you don’t have build problems with your modules that have a C or C++ component, use it. It is small, fast and well vetted.
Note: There are workarounds for some of the more common non-pure-python module build failures but usually involve much shenaniganry that I have neither the time nor inclination to pursue.
The slim images are similar to the ubuntu images. Rather than an Ubuntu base image, they use a Debian 8 base.
| REPOSITORY | TAG | SIZE |
|---|---|---|
| python | 3.6.3 | 690MB |
| python | 3.6-slim | 201MB |
| python | 3.6.3-alpine | 89MB |
| registry.gitlab.com/revsys/docker-builds/python | 3.6.3-wee | 153MB |
We settled on a Debian 8 image provided by google’s GCE service. It is smaller by way of excluding systemd-related bits while building the initial bootstrap chroot.
The resulting images are smaller than -slim but larger than alpine.
Optimized Python images
Three builds are available per Python version:
- :wee - stock build without compile or link-time optimizations enabled
- :wee-optimized - build with profile-guided optimization flag
- :wee-optimized-lto - build with PGO + Link-Time-Optimization flags
| REPOSITORY | TAG | SIZE |
|---|---|---|
| revolutionsystems/python | 3.5.3-wee-optimized-lto | 164MB |
| revolutionsystems/python | 3.5.3-wee-optimized | 164MB |
| revolutionsystems/python | 3.5.3-wee | 158MB |
Building
The registry is included from /.versions
make targets:
- build-image: docker build...
- push-image: docker push ...
- image: runs build-image then push-image
stock build
:# LTO= OPTIMAL= PYTHON_VERSION=3.5.3 make image
PGO optimizations
:# LTO= OPTIMAL=1 PYTHON_VERSION=3.5.1 make image
LTO Optimizations
:# LTO=1 OPTIMAL=1 PYTHON_VERSION=3.6.3 make image
Python 2 builds
The build arguments have stayed fairly consistent at least since Python 2.6. The build will not fail if the Python 3-specific environment variables remain set; however, the image will get a misleading tag. For Python 2 builds, use:
:# LTO= OPTIMAL= PYTHON_VERSION=2.7.13 make image
Build Optimization
Build times are radically reduced by using a local proxy. You can find an example squid 3/4 configuration in ./examples.
After setting the cache_dir path the command, squid -f /path/to/config -N -z
to initialize the cache filesystem storage.
To kick off the cache service: squid -f /path/to/config -N -a 3128.
Once the proxy is running, set the environment variable export PKG_PROXY=http://10.129.1.1:3128.
Docker transparently passes the HTTP_PROXY and HTTPS_PROXY environment variables
into each build container if they are configured with the --build-args flag.
The transparency is significant in that whether or not your build is leveraging a proxy will not affect the validity of the build host’s cache.
Origin Notes
Based in large part on the contents of the Docker Library Python repository.
- uses GNU make and environment variables to set options
LTO: if present and not empty, activate link-time-optimization functionalityOPTIMAL: if present and not empty, activate profiler-guided-optimization functionality- `PYTHON_VERSION’: e.g.: 3.6.3, 3.5.1
- Does not include TCL/TK. This is on purpose. If you neeed IDLE or pexpect you’ll need to customize the Dockerfile and rebuild.
- Uses multi-layer Dockerfile syntax
Keep in touch!
If you have a question about this project, please open a GitHub issue. If you love us and want to keep track of our goings-on, here's where you can find us online:
