Package your python app with poetry & docker

What the world might look like from inside a container.
Credit: Peter Burdon.

The combination of "package", "python", "poetry" and "docker" are words you may not hear very often these days as there is more than one way to skin a cat. Today, I choose to go off the beaten track of pip et al just for the fun of it all and if that gives you the heebie-jeebies you may try a chill-pill or please leave now.

三 THREE

二 TWO

一 ONE


〇 YES!

You are still here! Firstly congratulations and secondly, here is your TLDR copy and paste example:

FROM python:3.9-slim as os-base

ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1
RUN apt-get update
RUN apt-get install -y curl

FROM os-base as poetry-base

RUN curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
ENV PATH="/root/.poetry/bin:$PATH"
RUN poetry config virtualenvs.create false
RUN apt-get remove -y curl

FROM poetry-base as app-base

ARG APPDIR=/app
WORKDIR $APPDIR/
COPY your_project ./your_project
COPY pyproject.toml ./pyproject.toml
RUN poetry install --no-dev

FROM app-base as main

CMD tail -f /dev/null
Contents of Dockerfile

As per usual, should you require some unpacking, carry on reading.

This is called a multi-stage build and it offers a "significant reduction in complexity" by saving named interim stages that are used as a build cache and can be referenced internally. Let's check it out stage by stage:


Build Stage 1: os-base

FROM python:3.9-slim as os-base

ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1
RUN apt-get update
RUN apt-get install -y curl
Stage 1: os-base
  1. Named stage.
  2. PYTHONUNBUFFERED forces the stdout and stderr streams to go straight to terminal.
  3. PYTHONDONTWRITEBYTECODE prevents Python from writing .pyc files to disk. This is useful inside a docker container as a marginal performance gain if your process runs only once and needs not be saved to byte code for subsequent invocations.
  4. "... used to resynchronise the package index files from their sources", therefore does not hurt to run.
  5. We need curl for installing poetry in the next step.

Build Stage 2: poetry-base

FROM os-base as poetry-base

RUN curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
ENV PATH="/root/.poetry/bin:$PATH"
RUN poetry config virtualenvs.create false
RUN apt-get remove -y curl
Stage 2: poetry-base
  1. Named stage.
  2. Installing poetry as per the install instructions.
  3. Docker variables declared with ENV can be used across all docker stages. Here we're modifying our PATH environment variable so that we can execute poetry here and in subsequent stages.
  4. Since we are inside a docker container there is no immediate reason to use a virtual environment. As with most things, if you put a lot of software engineers in a room there will potentially be a few opinions on that. However, since I have come across scenarios where a virtual environment is required to prevent conflicts with the OS, I appreciate all experiences and leave it to you. If you choose to use a virtual environment, make use of poetry's fabulous ability to switch between versions/environments.
  5. Since we have no use for curl anymore, it doesn't hurt to remove it. In case you're wondering what the "-y" flag does, it's answering "yes" if the uninstaller asks you a question such as "are you a happy developer?". Not that it will but in a parallel universe it could. If you want to have some fun ask yourself "am I a happy developer?" and type "yes" in your linux terminal.

Build Stage 3: app-base

Here's where you'll copy your app specific files and directories. I left it quite open so you can replace it with your own.

FROM poetry-base as app-base

ARG APPDIR=/app
WORKDIR $APPDIR/
COPY your_project ./your_project
COPY pyproject.toml ./pyproject.toml
RUN poetry install --no-dev
Stage 3: app-base

By now you should be able to decipher what the steps do. I will slightly freestyle without numbering then, shall I?

WORKDIR does what it says on the tin. It's also magically created if it doesn't exist.

Notice that we are only copying pyproject.toml and NOT the poetry.lock. Why don't you find out why that might or might not be a good idea?

And next comes, the mighty dependency installation minus the development dependencies. You have hopefully run your pytest tests during your build pipeline if you haven't and you are interested in the Google Cloud Platform, join me next time to see how to automate unit and integration testing for your python project with Cloud Build.


Epilogue - Build Stage 4: main

This is where you launch your app. We're not launching an app in this tutorial but using the space to show you a little trick instead.

FROM app-base as main

CMD tail -f /dev/null
Fun tail

Your container will now remain running. You can either ssh to it to e.g check manually the dependencies have been installed, or what the world might look like from inside a container.

じゃまたね or じゃまた or またね