Once "It works on my machine™", what do I do with my Python program?
If it's a lib, share it on Pypi. If it's a program for an end user, create an executable.
And if it's a web service, deploy it on a server.
Now there is no universal story for the latter, but you need to know how to replicate your setup on the distant machine, particularly the dependencies in your virtualenv.
Most projects have some way to push the code on the machine and then trigger the installation of the new venv.
For example, a revolutionary bitecode-home-page-size micro-service project could be a
hello_word.py
file that looks like this:
import requests
print(len(requests.get("https://www.bitecode.dev").content))
But it needs
requests
, how do you get that on the server?
One way is of course to
pip freeze
then
pip install -r
, or whatever is the equivalent of your packaging tool of choice.
Another way is to gulp the
uv
hype and do:
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "requests",
# ///
import requests
print(len(requests.get("https://www.bitecode.dev").content))
Then on the server, install
uv
so you can
uv run
.
But what if the machine doesn't have access to Pypi? Or what if you want to create a single artifact, push that on the server, and make it just work "as-is", like a
*.war
file in Java?
There is a standard format for that in Python, called the
zipapp
, which is essentially an executable zip file containing all your code and its dependencies.
Several tools, such as the stdlib
zipapp module
or
pex
, can build one. But the first program is limited in features, while the second one doesn't work on Windows.
Today, we are going to look at
shiv
, which is what I currently use for this kind of task.
Shiv does only one thing: take your code and turn it into a zipapp.
But it comes with two tricks:
-
The first run, the zipapp will decompress into a cache directory. This means the first run is slow, but the next ones are faster. It also allows you to use code that is not zip-safe, like a Django project that wants to read static files in its own source using
__file__
.
-
You can give it a script to execute before your code runs but after the decompression, if you need some setup to take place.
Unfortunately, shiv's doc is not great, and there are several ways to use it, none being particularly obvious.
It also requires you to understand well the process because you will likely want to tweak it, so it's not transparent magic that you can run and forget.
More than that, if your dependencies contain compiled extensions, the zipapp will only work on the same type of OS + architecture than the one you created the zipapp on. You can make a zipapp on windows and run it on linux with a bit of a hack, but I wouldn't advise you to do it (use WSL instead).
Finally, zipapps are not standalone executables like you could produce with pyinstaller or nuitka. You still need Python installed on the target machine.
Still, it's a nice tool once it's set up correctly, as it lets you to have one big file you just drop on the server.
Let's create a venv,
pip install shiv requests
in it, and then let's go back to our original
hello_word.py
script:
import requests
if __name__ == "__main__":
print(len(requests.get("https://www.bitecode.dev").content))
There are several ways to use
shiv
, with a console script, or with a Python module. I find using the Python module easier and more robust. For this, we can turn our script into a well-mannered entry point:
import requests
def main():
print(len(requests.get("https://www.bitecode.dev").content))
if __name__ == "__main__":
main()
This way, our script can be imported without side effects, and we can point to the function
main()
using the import path
hello_word.main
to say “this is how our program starts”.
You can put any code here: an
argparse
declaration, the code of Django's
manage.py
, your entire program, or just a few imports and one call. It doesn't matter, the important thing is that it's the main entry point.
Then we create the zipapp with:
cp <MODULE> <PATH_TO_PACKAGES> # don't forget this or your code won't be in the zip
shiv --site-packages <PATH_TO_PACKAGES> -e <MODULE_ENTRY_POINT> -o <ZIPAPP_NAME>
E.G:
cp hello_word.py .venv/lib/python3.11/site-packages/
shiv --site-packages .venv/lib/python3.11/site-packages/ -e hello_word.main -o hello_word.pyz
In my case, I directly passed
.venv/lib/python3.11/site-packages/
, which is the directory where all the packages are installed by pip in my venv. It's different depending on the tooling and the platform. Note that you don't even have to use a venv,
shiv
only wants a directory of stuff to copy, you can pass any directory that contains code.
In fact, I usually create a separate dir and copy both my entry point and my venv packages in there, it's cleaner.
hello_word.main
tells
shiv
it can run the entry point by doing
import hello_word; main()
.
The goal of all this, the output, will be a zip file named
hello_word.pyz
.
You can then run the zipapp with Python:
❯ python hello_word.pyz
62236
Our revolutionary script has successfully run. You can even see that shiv uncompressed it transparently:
❯ ls ~/.shiv
hello_word.pyz_2ecc0aa8cd46b0ed00ffc897ca8e009664adb36582f63c85274c2f0f2fceb64d
All files in the
--site-packages
directory will be copied in the zip, Python or not, so you can put them there before building the zipapp. Now, if you don't want to pollute your venv with this, create a temp dir with all the things you need: packages, static files, whatever.
It's a good practice anyway.
However, how do you read them? After all, they will be on a path like:
~/.shiv/hello_word.pyz_2ecc0aa8cd46b0ed00ffc897ca8e009664adb36582f63c85274c2f0f2fceb64d/goeste.jpg
If they are inside a dependency package, it's not a problem. However, it's my experience that most users use
shiv
with projects that are not packages, such as web projects, for which they have tons of static files they want to use.
For this, I recommend simply copying the files you need using a preamble script, which I will explain below.
When you execute your zipapp, the code is actually decompressed and then run from a directory inside
~/.shiv
(or anything you configured).
But this can be a problem if you have things that are path-dependent, like:
-
Sqlite DBs.
-
Static assets served from apache/nginx/caddy.
-
CSV/JSON/YAML config files your code wants to read assuming it could do
pathlib.Path(__file__).open()
.
-
Commands to migrate your projects in prod, like
manage.py collectstatic/migrate/makemessages
in Django.
Since
shiv
can't find a generic solution to all those problems, it instead provides you with hooks to deal with this.
The simplest one is
--build-id TEXT
that you can pass when you build your zipapp. You can set the ID to whatever you want, BUT IT MUST BE UNIQUE PER BUILD.
Good candidates are timestamps, git commit hashes, or even a release version:
❯ shiv --site-packages .venv/lib/python3.11/site-packages/ -e hello_word.main -o hello_word.pyz --build-id 3.1.2
❯ python hello_word.pyz
62236
❯ ls ~/.shiv/
hello_word.pyz_2ecc0aa8cd46b0ed00ffc897ca8e009664adb36582f63c85274c2f0f2fceb64d
hello_word.pyz_3.1.2
This way the directory becomes predictable, and you can configure the rest of your stack based on this.
If you prefer for automation, dynamism, or simply need something more custom, you can use a preamble script, which is a separate Python script
shiv
will call after decompression is confirmed, but before your project code runs.
It's a special script because
shiv
magically injects 3 variables in there:
-
archive
: the path to the zip of zipapp.
-
site_packages
: the path to the decompressed directory.
-
env
: a dict of metadata about the zipapp.
Here is an example of
preamble.py
script that prints those information:
import pprint
print('preamble starts!')
print(archive)
print(site_packages)
pprint.pprint(vars(env))
print('preamble ends!')
Note that I don't need to import those variables, they are automatically present in the preamble script.
Then I build my zipapp with it:
❯ shiv --site-packages .venv/lib/python3.11/site-packages/ -e hello_word.main -o hello_word.pyz --preamble preamble.py
❯ python hello_word.pyz
preamble starts!
./hello_word.pyz
/home/user/.shiv/hello_word.pyz_cee74e724d5354fa7c04798ae7424b041b944d91d5e3f1f614d49bdb87947263
{'_compile_pyc': False,
'_entry_point': 'hello_word.main',
'_extend_pythonpath': False,
'_prepend_pythonpath': None,
'_root': None,
'_script': None,
'always_write_cache': False,
'build_id': '3.1.2',
'built_at': '2025-01-01 17:41:57',
'hashes': {},
'no_modify': False,
'preamble': 'preamble.py',
'reproducible': False,
'shiv_version': '1.0.8'}
preamble ends!
62236
You can put whatever you want in this script, like moving or copying files around, running db migrations, calling systemd commands, etc.
I tend to put things like cleaning up old decompressed directories, moving static files into a directory that nginx can serve, or calls to
chmod
.
The preamble will run
every time
, so make sure you make it idempotent.
-
uv run
works with zipapp.
-
You can change the place where
shiv
will decompress the files using
--root
. Shiv will make this information available through the env var
SHIV_ROOT
to the zipapp code.
-
You don't really need the preamble, it's more of a convenience thing. You can replicate it roughly by doing this in your entry point:
import os
import sys
import zipfile
from pathlib import Path
from _bootstrap.environment import Environment
SHIV_ROOT = Path(os.environ.get("SHIV_ROOT", "~/.shiv")).expanduser().absolute()
ZIPAPP_NAME = sys.argv[0]
with zipfile.ZipFile(ZIPAPP_NAME) as archive:
env = Environment.from_json(archive.read("environment.json").decode())
extraction_dir = SHIV_ROOT / f"{ZIPAPP_NAME}_{env.build_id}"
E.G, a minimalist
hello_django.py
production entry point using
waitress
:
import os
import sys
import django
from django.core.management import execute_from_command_line
from waitress import serve
def main():
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings")
django.setup()
if len(sys.argv) > 1 and sys.argv[1] == "runserver":
from project.wsgi import application
# you probably want to put this in env vars (and also DEBUG in settings)
serve(application, host="127.0.0.1", port=8000)
else:
execute_from_command_line(sys.argv)
if __name__ == "__main__":
main()
Now if I do:
> source .venv/bin/activate
> pip install django waitress # or -r requirements.txt
> cp -r project .venv/lib/python3.11/site-packages/ # add the django project to the zipapp
> cp -r hello_django.py .venv/lib/python3.11/site-packages/
> shiv --site-packages .venv/lib/python3.11/site-packages/ -e hello_django.main -o hello_django.pyz
I can run the
waitress
server in production this way:
> python hello_django.pyz runserver
Or do things like run migrations this way:
> python hello_django.pyz migrate
This is definitely not straightforward and doesn't even deal with other issues like the static files
Therefore,
shiv
, while allowing us to achieve our goal, is still an expert tool at this stage where you need to understand the underlying layer to make sure it works well.