Byron Peebles – Compiling Python using pyenv with optimizations

4 months ago 2

I noticed awhile ago that you can pass build arguments to pyenv when installing a new version of Python. I had also recently seen a mention of building Python targeted toward the native architecture, as well as Link Time Optimization (LTO). By setting PYTHON_CFLAGS and/or CONFIGURE_OPTS by doing something like:

PYTHON_CFLAGS='-march=native -mtune=native' CONFIGURE_OPTS='--enable-optimizations --with-lto' pyenv install 3.10.2

So I decided to try it and benchmark using my sudoku solver.

Unoptimized

$ pyenv install 3.10.2 $ pyenv shell 3.10.2 $ time python sudoku.py 850002400720000009004000000000107002305000900040000000000080070017000000000036040 859612437723854169164379528986147352375268914241593786432981675617425893598736241 real 1m24.791s user 1m24.770s sys 0m0.021s $ time python sudoku.py 850002400720000009004000000000107002305000900040000000000080070017000000000036040 859612437723854169164379528986147352375268914241593786432981675617425893598736241 real 1m25.261s user 1m25.252s sys 0m0.009s

So, an average real time of 85 seconds across two runs.

Optimized

$ PYTHON_CFLAGS=-march=native CONFIGURE_OPTS='--enable-optimizations --with-lto' pyenv install 3.10.2 $ pyenv shell 3.10.2 $ time python sudoku.py 850002400720000009004000000000107002305000900040000000000080070017000000000036040 859612437723854169164379528986147352375268914241593786432981675617425893598736241 real 1m6.683s user 1m6.679s sys 0m0.004s $ time python sudoku.py 850002400720000009004000000000107002305000900040000000000080070017000000000036040 859612437723854169164379528986147352375268914241593786432981675617425893598736241 real 1m5.037s user 1m5.032s sys 0m0.005s

An average real time of 66 seconds, so only taking about 79% of the time of unoptimized Python. I’ve also noticed a large unit test suite that significantly uses Postgres as well and still saw it complete 5–10% faster. Building the Python took significantly more time (didn’t time it) but I only build a version of Python once and then run it many many times.